A Bidirectional Cross Spatiotemporal Fusion Network with Spectral Restoration for Remote Sensing Imagery

Abstract

Existing deep learning-based spatiotemporal fusion (STF) methods for remote sensing imagery often focus exclusively on capturing temporal changes or enhancing spatial details while failing to fully leverage spectral information from coarse images. To address these limitations, we propose a Bidirectional Cross Spatiotemporal Fusion Network with Spectral Restoration (BCSR-STF). The network integrates temporal and spatial information using a Bidirectional Cross Fusion (BCF) module and restores spectral fidelity through a Global Spectral Restoration and Feature Enhancement (GSRFE) module, which combines Adaptive Instance Normalization and spatial attention mechanisms. Additionally, a Progressive Spatiotemporal Feature Fusion and Restoration (PSTFR) module employs multi-scale iterative optimization to enhance the interaction between high- and low-level features. Experiments on three datasets demonstrate the superiority of BCSR-STF, achieving significant improvements in capturing seasonal variations and handling abrupt land cover changes compared to state-of-the-art methods.

1. Introduction

Spatiotemporal Fusion (STF) integrates remote sensing data from different sensors and time points [1,2,3], aiming to generate time series remote sensing data with high temporal and spatial resolution. STF effectively addresses the trade-off between temporal and spatial resolution in remote sensing imagery, making it widely applicable in areas such as agricultural monitoring [4,5,6], ecosystem dynamics research [7], and urban heat island analysis [8,9]. Traditional methods, including those based on unmixing [10,11,12], weighted functions [13,14,15], Bayesian [16,17], and machine learning techniques [18,19], have established the theoretical foundation for STF. While these methods are simple and unsupervised, their reliance on linear assumptions limits their ability to model complex remote sensing features.

In recent years, deep learning (DL), with its powerful nonlinear modeling capabilities and automated feature learning, has transformed the research paradigm of STF. Convolutional neural networks (CNNs) are the most widely adopted for STF applications among diverse neural network designs. Models such as DCSTFN [20] and EDCSTFN [21] employ fully end-to-end architectures, using separate CNN branches to extract fine and coarse features. Generative adversarial networks (GANs) address the issue of overly smoothed fusion results, often encountered when training solely with CNNs, by leveraging adversarial loss. For example, GAN-STFM [22] integrates conditional GANs and switchable normalization techniques into STF. Similarly, MLFF-GAN [23] incorporates U-Net-like multi-level feature fusion into the GAN framework. Both approaches use CNNs as the backbone for generator and discriminator networks. However, CNNs struggle to capture long-range dependencies in large-scale spatiotemporal data and adapt to significant heterogeneity due to their fixed weight-sharing mechanism. Transformers [24,25], which utilize self-attention mechanisms for dynamic weight adjustment, have been applied to STF to overcome these limitations. For instance, SwinSTFM [26] effectively combines window-based self-attention [27] with linear unmixing theory, significantly improving spatiotemporal feature modeling capabilities. Additionally, several studies combine CNNs and Transformers to exploit their complementary strengths. STF-Trans [28] employs CNNs as shallow feature extractors and Vision Transformers for deep modeling of long-range dependencies. CTSTFM [29] introduces spatial and channel attention modules along with cross-attention mechanisms to achieve efficient feature extraction and fusion.

In addition to the selection and optimization of backbone networks, DL-based STF methods can also be examined in terms of their core tasks: temporal change capture or spatial detail enhancement. DL methods focused on temporal changes primarily analyze the relationship between images captured at different time points. These methods emphasize capturing variation features within time series and constructing fine-grained difference features. Some approaches directly utilize coarse-difference images and fine images as input. For instance, StfNet [30] adds the coarse-difference images and fine images before feeding them into a CNN-based fusion network. Similarly, PDCNN [31] and STFMCNN [32] stack these inputs for processing through the fusion network. DMNet [33], AMNet [34], and STFDSC [35] extract features from input data using CNNs and subsequently stack or sum these features before passing them to the fusion network. Additionally, networks like DCSTFN [20], EDCSTFN [21], and PSTAF-GAN [36] directly process sequential coarse images, employing difference modeling on the extracted coarse features to enhance temporal fusion.

In contrast, DL methods focusing on spatial details emphasize the scale relationship between coarse and fine images captured on the same date. These methods often approach the STF task as a super-resolution (SR) problem, where the resolution of the coarse image is enhanced to generate a high-resolution image with rich details. For example, STFDCNN [37] employs a five-layer super-resolution CNN to model the nonlinear relationship between MODIS and Landsat images, combining high-pass modulation to fully exploit information from the reference image. Furthermore, some STF networks align with reference-based SR tasks. Notable examples include GAN-STFM [22], MCBAM-GAN [38], STF-Trans [28], and CTSTFM [29], which directly input fine images from the reference time and coarse-resolution images from the target time. These approaches achieve high-accuracy spatiotemporal fusion while reducing dependence on the number of input images.

However, in highly heterogeneous regions, particularly when significant land cover changes occur between prediction and reference dates, models face the dual challenge of capturing temporal variations and preserving spatial structures. Some models address this challenge through parallel processing; for instance, DL-SDFM [39] employs dual CNN architectures to independently predict temporal changes and spatial information before fusing them using weighting functions. MTDL-STF [40] utilizes separate SR and STF networks to predict normalized difference vegetation index (NDVI) values along spatial and temporal dimensions, respectively. However, such decoupled approaches often overlook the intricate interdependencies between temporal and spatial scales, resulting in inaccurate mapping of temporal changes across different scales. Furthermore, significant land cover changes imply that the spectral information from the coarse image at the prediction date may be the only reliable source for predicting the spectra of new land cover types. However, most existing methods fail to fully utilize the prior spectral information provided by coarse images, leading to spectral distortion and insufficient consistency in the fusion results.

Based on the above research, a Bidirectional Cross Spatiotemporal Fusion Network with Spectral Restoration (BCSR-STF) is proposed. This framework achieves bidirectional fusion for temporal change modeling (time direction) and detail enhancement modeling (scale direction) through a Progressive Spatiotemporal Feature Fusion and Restoration (PSTFR) module. The PSTFR module incorporates a multi-scale optimization design with local and global attention. At each level, it enables bidirectional spatiotemporal feature interaction across short-range and long-range distances, and it adaptively optimizes the fused features using spectral prior knowledge. The contributions of this work are summarized as follows:

• An end-to-end BCSR-STF model is proposed. The PSTFR module within BCSR-STF employs multi-scale iterative optimization, enabling effective exchange between high-level and low-level information. The design enhances the model’s capability to address variations in object scale within input images, thereby improving the accuracy of spatiotemporal fusion.
• A Bidirectional Cross Fusion (BCF) module is designed to leverage the advantages and mitigate the limitations of temporal and scale directions. This module simultaneously considers temporal variations and scale differences, utilizing short-range and long-range attention mechanisms based on the Vision Transformer to enhance interactions between temporal and spatial information, thereby improving fusion accuracy.
• The Global Spectral Restoration and Feature Enhancement (GSRFE) module is introduced to restore and enhance spectral information often overlooked in coarse images. By incorporating Adaptive Instance Normalization (AdaIN) and spatial attention mechanisms, GSRFE adaptively adjusts spectral distributions and enhances the quality of spatiotemporal fusion through feature enhancement.

The remainder of this work is structured as follows: Section 2 provides a detailed explanation of the framework and the specifics of the proposed network. Section 3 provides the experimental analyses. Section 4 discusses the effectiveness and limitations of the methodology. Finally, Section 5 concludes this work.

2. Methodology

2.1. Network Architecture

The overall framework of the proposed BCSR-STF is illustrated in Figure 1. It comprises three primary components: feature extraction, feature fusion, and a prediction head. The feature extraction module employs a pyramid structure to facilitate multi-scale and multi-level feature representation. The feature fusion module consists of multi-level PSTFR to iteratively refine the fused features, enhancing detail and ensuring coherence across scales. Lastly, the prediction head reconstructs the refined features into a fused image, producing the desired output. The model operates with three input images: $C_{t_0}$, $F_{t_0}$, and $C_{t_1}$. Here, $C_{t_0}$ and $F_{t_0}$ denote the coarse and fine images in the reference data, respectively, while $C_{t_1}$ refers to the coarse image in the predicted data. $\widehat{F_{t_1}}$ is the network output, i.e., the fine image in the predicted data.

In the feature extraction stage, a multi-scale patch embedding strategy [41] is first employed to transform the input images $C_{t_0},C_{t_1}$, and $F_{t_0}$ into serialized tokens. This approach utilizes convolutional kernels of four sizes (2 × 2, 4 × 4, 8 × 8, and 16 × 16), all with a fixed stride of 2, to sample and embed features at varying spatial scales. Smaller kernels emphasize fine-grained local details, while larger kernels capture broader contextual information. The resulting features are embedded into a unified token sequence, preserving multi-scale spatial information for subsequent processing.

$$L_0^0={PE}_{{\omega }_1}\left(C_{t_0}\right)$$

(1)

$$L_1^0={PE}_{{\omega }_1}\left(C_{t_1}\right)$$

(2)

$$H_0^0={PE}_{{\omega }_2}\left(F_{t_0}\right)$$

(3)

${PE}_{\omega }\left(·\right)$ represents the multi-scale patch embedding module parameterized by $\omega$. The feature extractors for $C_{t_0}$ and $C_{t_1}$ are based on a Siamese Network, which is used to assess the similarities and differences between $C_{t_0}$ and $C_{t_1}$. The low-resolution features extracted from $C_{t_0}$ and $C_{t_1}$ are represented as $\left\{L_0^{\left(1\right)},\dots ,L_0^{\left(i\right)},L_0^{\left(i+1\right)}\dots L_0^{\left(N\right)}\right\}$ and $\left\{L_1^{\left(1\right)},\dots ,L_1^{\left(i\right)},L_1^{\left(i+1\right)}\dots L_1^{\left(N\right)}\right\}$, respectively. The resolution of these features progressively decreases from the finest level 1 to the coarsest level $N$, where the size of the current-level feature $L_t^{\left(i\right)}$ is twice that of the feature at the next level $L_t^{\left(i+1\right)}$. Similarly, the high-resolution features extracted from $F_{t_0}$ are represented as $\left\{H_0^{\left(1\right)},\dots ,H_0^{\left(i\right)},H_0^{\left(i+1\right)}\dots H_0^{\left(N\right)}\right\}$. Formally, we define

$$L_t^{\left(i\right)}=P_{{\varphi }_1}^{\left(i\right)}\left(E_{{\theta }_1}^{\left(i\right)}\left(L_t^{\left(i-1\right)}\right)\right)$$

(4)

$$H_0^{\left(i\right)}=P_{{\varphi }_2}^{\left(i\right)}\left(E_{{\theta }_2}^{\left(i\right)}\left(H_0^{\left(i-1\right)}\right)\right)$$

(5)

where $t$ represents the image acquisition time. $E_{{\theta }_1}^{\left(i\right)}\left(·\right)$ represents the low-resolution feature extraction module parameterized by ${\theta }_1$ at level $i$. $E_{{\theta }_2}^{\left(i\right)}\left(·\right)$ represents the high-resolution feature extraction module parameterized by ${\theta }_2$. $P_{\varphi }(\cdot )$ denotes the parameterized Patch Merging operation [27], which performs downsampling by concatenating neighboring patches and applying a learnable linear transformation to reduce the dimensionality of the resulting features. The feature extraction module employs the CrossFormer Block [41], which facilitates multi-scale interactions between neighboring embeddings and distant embeddings, effectively capturing local details and global contextual information.

To address the scale variability in remote sensing imagery and the resolution differences between coarse and fine images, a multi-level distributed fusion module is proposed for the feature fusion stage. Each fusion stage facilitates the interaction between high-level and low-level features through multi-scale iterative optimization, allowing the fused features to progressively transition from low resolution to high resolution. This process involves upsampling, implemented using the Pixel Shuffling technique, followed by PSTFR and the fusion of high-level and low-level features through the CrossFormer Block.

PSTFR is the core component of the proposed BCSR-STF model. It comprises two primary modules: (1) Bidirectional Cross Fusion (BCF), which performs spatiotemporal feature fusion along both time and scale directions to address the limitations of single-direction fusion, and (2) Global Spectral Restoration and Feature Enhancement (GSRFE), which utilizes spectral correlations in coarse images for spectral recalibration and improved feature fusion. The overall process is outlined as follows:

$$F_{st}^{\left(i\right)}=\mathrm{B}\mathrm{C}\mathrm{F}(L_0^{\left(i\right)},L_1^{\left(i\right)},H_0^{\left(i\right)})$$

(6)

The fused spatiotemporal feature $F_{st}^{\left(i\right)}$ is input along with the low-resolution fine features $L_1^{\left(i\right)}$ into the GSRFE module. The goal of this module is to repair and enhance the spatiotemporal fusion features by utilizing spectral information from the low-resolution imagery, ensuring that details are preserved during the fusion process.

$$F_{s{t}_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}}}^{\left(i\right)}=\mathrm{G}\mathrm{S}\mathrm{R}\mathrm{F}\mathrm{E}(F_{st}^{\left(i\right)},L_1^{\left(i\right)})$$

(7)

After obtaining the repaired and enhanced spatiotemporal features $F_{s{t}_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}}}^{\left(i\right)}$, they are further fused with the output features $F_{s{t}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}}^{(i+1)}$ from the previous stage at different levels to ensure effective complementarity between high- and low-level features and the preservation of details.

$$F_{s{t}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}}^{\left(i\right)}=E_{{\theta }_3}^{\left(i\right)}\left(cat\left(F_{s{t}_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}}}^{\left(i\right)},PS\left(F_{s{t}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}}^{(i+1)}\right)\right)\right)$$

(8)

Here, $PS\left(·\right)$ denotes the Pixel Shuffling upsampling technique, and $cat\left(·\right)$ represents concatenation. After the final fusion stage, the last fused feature, $F_{s{t}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}}^{\left(0\right)}$, is passed to the prediction head to reconstruct the fused image. This process begins with Pixel Shuffling to restore the feature map to its original dimensions, followed by two convolutional layers to produce the reconstructed fused image.

2.2. Bidirectional Cross Fusion

The BCF module is designed to effectively integrate spatiotemporal features across both time and scale dimensions, enhancing temporal consistency and preserving details in remote sensing images. Unlike existing single-direction fusion modules, the BCF module captures temporal change and spatial details simultaneously, enabling bidirectional feature interactions. This approach improves robustness in scenarios with complex land cover changes. As shown in Figure 2, the module employs feature fusion strategies based on self-attention mechanisms for both temporal and scale dimensions. Furthermore, it integrates short-distance attention (SDA) and long-distance attention (LDA) [41] modules to reinforce local and global feature interactions.

2.2.1. Time Direction

The time direction primarily focuses on optimizing the fused features through the temporal changes across time points. In the feature fusion of the time direction, the idea from STARFM is adopted. By fusing the fine features from the reference time with the difference features generated by temporal changes, the fused spatiotemporal features are obtained. The mathematical expression is

$$F_t^{\left(i\right)}=H_0^{\left(i\right)}+f_{time}\left(H_0^{\left(i\right)},L_0^{\left(i\right)},{∆L}^{\left(i\right)}\right)$$

(9)

where $F_t^{\left(i\right)}$ is the spatiotemporal feature fused along the time direction and $f_{time}\left(·\right)$ represents the feature fusion operation in the time direction. This operation includes a combination of short-range and long-range attention mechanisms, followed by nonlinear feature mapping through a multi-layer perceptron (MLP) to enhance interactions between features.

The function $f_{time}\left(·\right)$ is introduced in detail below. The input feature maps are $H_0^{\left(i\right)},L_0^{\left(i\right)}$, and ${∆L}^{\left(i\right)}$. ${∆L}^{\left(i\right)}$ represents the temporal variation information. Specifically, ${∆L}^{\left(i\right)}$ = $L_1^{\left(i\right)}-L_0^{\left(i\right)}$, which denotes the feature difference between the reference time and the predicted time. It is used to guide the recovery and enhancement of fine-difference features in subsequent processes. A linear layer is applied to $H_0^{\left(i\right)}$,$L_0^{\left(i\right)}$, and ${∆L}^{\left(i\right)}$, mapping them to the query vector $Q_t^{\left(i\right)}$, key vector $K_t^{\left(i\right)}$, and value vector $V_t^{\left(i\right)}$.

$$Q_t^{\left(i\right)}=H_0^{\left(i\right)}{W}_{Q_t}^{\left(i\right)},K_t^{\left(i\right)}=L_0^{\left(i\right)}{W}_{K_t}^{\left(i\right)},V_t^{\left(i\right)}={∆L}^{\left(i\right)}{W}_{V_t}^{\left(i\right)}$$

(10)

The matrices $W_{Q_t}^{\left(i\right)}$,$W_{K_t}^{\left(i\right)}$, and $W_{V_t}^{\left(i\right)}$ are the learnable weight parameters. The calculation formula for the attention mechanism is

$$Attention(Q_t^{\left(i\right)},K_t^{\left(i\right)},V_t^{\left(i\right)})=SoftMax\left({\displaystyle \frac{Q_t^{\left(i\right)}{K_t^{\left(i\right)}}^T}{\sqrt{d}}}+B\right)V_t^{\left(i\right)}$$

(11)

The attention mechanism is a crucial part of $f_{time}\left(·\right)$. The inner product of $Q_t^{\left(i\right)}$ and ${K_t^{\left(i\right)}}^T$, normalized by $\sqrt{d}$ (where $d$ is the dimensionality of the feature vectors) determines the attention weights. These weights are then passed through $SoftMax\left(·\right)$ to ensure they sum to one, effectively distributing attention across the features. A bias term $B$ is added to introduce positional information. The weighted sum of $V_t^{\left(i\right)}$ is computed, representing the aggregated temporal information enhanced by attention. The core idea of the entire process is to leverage the latent scale relationship between coarse image $H_0^{\left(i\right)}$ and fine images $L_0^{\left(i\right)}$ to dynamically adjust the coarse change features ${∆L}^{\left(i\right)}$, thereby enhancing the model’s ability to capture fine temporal variation information.

To reduce the computational cost of attention, a window-based attention mechanism is employed to achieve global interactions among pixels within each window. The window grouping methods for SDA and LDA are illustrated in Figure 3, where only two groups are shown for simplicity. Pixels enclosed by the red rectangles form one group, while those within the purple rectangles form another. As depicted in Figure 3a, the window grouping in SDA ensures that pixels within the same group are spatially adjacent. In contrast, the grouping method in LDA, shown in Figure 3b, follows a principle that enables non-adjacent pixel embedding. By concatenating SDA and LDA, both short-distance and long-distance feature interactions are effectively captured.

2.2.2. Scale Direction

In the feature fusion along the scale direction, the coarse features of the predicted time are combined with the high-frequency features that the reference time image can provide, enhancing the detailed information across different scales. The specific expression is as follows:

$$F_s^{\left(i\right)}=L_0^{\left(i\right)}+f_{scale}\left(H_0^{\left(i\right)},L_1^{\left(i\right)},L_0^{\left(i\right)}\right)$$

(12)

Here, $F_s^{\left(i\right)}$ is the spatiotemporal feature after fusion along the scale direction, and $f_{scale}\left(·\right)$ is the fusion operation along the scale direction, which utilizes an attention mechanism similar to that used in the time direction.

Similar to the design in the time direction, the attention mechanism in the scale direction is also based on the calculation of queries, keys, and values to establish the relationship of spatiotemporal features. The difference is that in the scale direction, the computation of queries, keys, and values is adjusted to accommodate the relationship between images at the same resolution from different time points. Specifically, we compute the following queries, keys, and values:

$$Q_s^{\left(i\right)}=L_0^{\left(i\right)}{W}_{Q_s}^{\left(i\right)},K_s^{\left(i\right)}=L_1^{\left(i\right)}{W}_{K_s}^{\left(i\right)},V_s^{\left(i\right)}=H_0^{\left(i\right)}{W}_{V_s}^{\left(i\right)}$$

(13)

$$Attention(Q_s^{\left(i\right)},K_s^{\left(i\right)},V_s^{\left(i\right)})=SoftMax\left({\displaystyle \frac{Q_s^{\left(i\right)}{K_s^{\left(i\right)}}^T}{\sqrt{d}}}+B\right)V_s^{\left(i\right)}$$

(14)

where $W_{Q_s}^{\left(i\right)}$, $W_{K_s}^{\left(i\right)}$, and $W_{K_s}^{\left(i\right)}$ are the learnable weight matrices for the $i$-th stage of scale direction fusion. By computing $Q_s^{\left(i\right)}{K_s^{\left(i\right)}}^T$, the potential relationship matrix between $L_0^{\left(i\right)}$ and $L_1^{\left(i\right)}$ is obtained. This matrix represents the potential differences between images of the same resolution at different time points. This difference is intended to be used to correct the high-frequency detail information obtained from $H_0^{\left(i\right)}$, which aids in guiding the detail enhancement of the coarse-resolution image.

Finally, the time-direction fused features $F_t^{\left(i\right)}$ and the scale-direction fused features $F_s^{\left(i\right)}$ are combined. The two feature sets are merged using a concatenation operation and processed through a linear transformation to obtain the final spatiotemporal fusion features $F_{st}^{\left(i\right)}$:

$$F_{st}^{\left(i\right)}=linear\left(cat(F_t^{\left(i\right)},F_s^{\left(i\right)})\right)$$

(15)

where $cat\left(·\right)$ refers to the concatenation operation along the feature dimension and $linear\left(·\right)$ represents a linear transformation along the feature dimension. This approach ensures that both time and scale direction information is preserved during the fusion process, resulting in spatiotemporal features with richer contextual information.

2.3. Global Spectral Restoration and Feature Enhancement

When the phenological changes in remote sensing scenes are quite evident, the spectral prior information of the fused image can only come from the coarse image $C_{t_1}$ at the reference time. Most existing STF models neglect the preservation of spectral information from $C_{t_1}$ during the fusion process. To improve the spectral consistency and detail restoration capability of the spatiotemporal fusion results, the GSRFE module was designed.

The purpose of GSRFE is to use the spectral information from the coarse imagery at the predicted time to correct and enhance the obtained spatiotemporal fusion features. It combines Adaptive Instance Normalization (AdaIN) and spatial attention mechanisms to enhance and repair the input features. By adaptively adjusting the spectral distribution of the features, it improves the quality of spatiotemporal feature fusion. The design process of the GSRFE module is shown in Figure 4, which mainly includes the following steps:

(1) Spectral information extraction and mapping: First, the coarse image feature $L_1^{\left(i\right)}$ at the reference time is globally average-pooled to obtain a global feature vector. After passing through a series of fully connected layers, the scaling and shifting parameters ${\lambda }^{\left(i\right)}$ and ${\beta }^{\left(i\right)}$ are learned, which are used to adjust the spectral distribution of the fused feature map.

(2) Spectral adaptive adjustment: First, the spatiotemporal fused feature $F_{st}^{\left(i\right)}$ undergoes channel normalization to obtain the normalized feature $F_{st}^{(i{)}^{\prime }}$. The learned parameters ${\lambda }^{\left(i\right)}$ and ${\beta }^{\left(i\right)}$ are then used to adjust the normalized feature.

$$F_{st}^{\left(i\right)″}{=\lambda }^{\left(i\right)}\times F_{st}^{\left(i\right)\prime }+{\beta }^{\left(i\right)}$$

(16)

(3) Spatial enhancement: To further correct the spectral inconsistencies caused by sensor system errors while retaining the detailed features from the spatiotemporal fusion information $F_{st}^{\left(i\right)}$, the corrected feature $F_{st}^{\left(i\right)″}$ is input into a spatial attention module. This module first applies average pooling and max pooling operations on the input features to obtain global information in the spatial dimension. These global features are then processed further through convolution operations, ultimately producing a spatial weight matrix $W_s^{\left(i\right)}$. The process can be expressed as

$$SA\left(x\right)=x\times \sigma \left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^1\left(\mathrm{L}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^3\left(\mathrm{c}\mathrm{a}\mathrm{t}\left(\mathrm{M}\mathrm{P}\left(x\right),\mathrm{A}\mathrm{P}\left(x\right)\right)\right)\right)\right)\right)$$

(17)

where $\sigma \left(·\right)$ denotes the Sigmoid activation function and ${\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^1\left(·\right)$ and ${\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^3\left(·\right)$ represent the operations of 1 × 1 convolution and 3 × 3 convolution, respectively. $\mathrm{M}\mathrm{P}\left(·\right)$ and $\mathrm{A}\mathrm{P}\left(·\right)$ refer to max pooling and average pooling, respectively.

Finally, the features $F_{st}^{\left(i\right)}$ and $F_{st}^{\left(i\right)″}$ enhanced by the attention module are added together to further enhance the spatial information. These enhanced features are refined through two 1 × 1 convolutions and activation functions, ultimately producing the globally spectral-restored and feature-enhanced feature $F_{s{t}_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}}}^{\left(i\right)}$.

$$F_{s{t}_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}}}^{\left(i\right)}={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^1\left(\mathrm{L}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^1\left(SA\left(F_{st}^{\left(i\right)}\right)+SA\left(F_{st}^{(i{)}^{″}}\right)\right)\right)\right)$$

(18)

2.4. Loss Function

The loss function used in our work is the sum of a pixel loss and a structural loss [26], expressed as

$$L=L_{\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}}+L_{\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{c}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{e}}$$

(19)

The pixel loss is based on the Charbonnier Loss, defined as

$$L_{\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}}=\sqrt{\parallel x-\widehat{x}{\parallel }^2+{ϵ}^2}$$

(20)

where $x$ and $\widehat{x}$ represent the ground truth and predicted images, respectively, and $ϵ$ is a small constant for numerical stability. This loss function optimizes pixel-level reconstruction accuracy and is robust to outliers, providing stability during optimization.

The structural loss is based on the Multi-Scale Structural Similarity (MS-SSIM) [21] index and is formulated as

$$L_{\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{c}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{e}}=1-\mathrm{M}\mathrm{S}\_\mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}(x,\widehat{x})$$

(21)

where $\mathrm{M}\mathrm{S}\_\mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}\left(x,\widehat{x}\right)$ evaluates the structural similarity at multiple scales, emphasizing both global and local image quality. It incorporates image information across multiple scales to comprehensively evaluate the structural similarity between two images.

3. Experimental Results

3.1. Study Areas and Datasets

In the experiments, three public datasets from different locations are used in this study—the Coleambally Irrigation Area (CIA) [42], the Lower Gwydir Catchment (LGC) [42], and the Wuhan [43].

The CIA study site is situated in southern New South Wales, Australia, and includes 17 pairs of cloud-free MODIS–Landsat images, each with a resolution of 6 × 1720 × 2040 pixels, collected between 2001 and 2002. This area is predominantly covered by rice fields. Although the land cover remained largely consistent during the dataset’s collection period, significant phenological variations were observed, making it an ideal dataset for evaluating the capability of BCSR-STF in capturing phenological changes.

The LGC study site, located in northern New South Wales, comprises 14 pairs of MODIS–Landsat cloud-free images, each measuring 6 × 2720 × 3200 pixels, acquired from 2004 to 2005. In mid-December 2004, the region experienced a major flood that inundated approximately 44% of the area. This abrupt event caused significant land cover changes, providing an excellent opportunity to evaluate the ability of BCSR-STF in predicting sudden transformations.

To explore the adaptability of BCSR-STF to different sensors, the Wuhan study site, located in an urban region of Hubei Province, China, is selected. This dataset includes eight pairs of Landsat–Gaofen images from 2015 to 2022, containing rich urban texture features and significant changes. The data size is 4 × 1000 × 1000.

3.2. Experiment Design and Evaluation

Three traditional STF methods (STARFM [13], FSDAF [11], and Fit-FC [15]) and five deep learning-based methods (EDCSTFN [21], GAN-STFM [22], MLFF-GAN [23], STF-Trans [28], and CTSTFM [29]) are used for comparison with the proposed method. All the mentioned approaches rely solely on a pair of reference images captured at different time points. These methods can be grouped based on their focus on either temporal change capture or spatial detail enhancement. STARFM, Fit-FC, EDCSTFN, and MLFF-GAN primarily target temporal difference modeling, leveraging input data from different time points to extract and reconstruct temporal features. On the other hand, GAN-STFM, STF-Trans, and CTSTFM emphasize spatial detail enhancement, using advanced architectures to generate high-resolution outputs by refining spatial features. FSDAF integrates temporal difference modeling and spatial detail enhancement by using linear unmixing theory to capture class changes between two time points and employing thin-plate spline interpolation for spatial prediction.

The datasets are divided into training and testing sets. For the CIA dataset, all MODIS–Landsat image pairs, except those from 25 November 2001, 12 January 2002, and 22 February 2002, are included in the training set. During testing, the image pair from 25 November 2001 and the coarse image from 12 January 2002 are used to predict the fine image for 12 January 2002. Similarly, for the LGC dataset, all images except those from 26 November 2004,12 December 2004, and 28 December 2004, are included in the training data. For testing, the image pair from 26 November 2004 and the coarse image from 12 December 2004 are used to predict the fine image for 12 December 2004. In the Wuhan dataset, all image pairs except those from 18 October 2015 and 30 October 2017 are allocated for training. For testing, 30 October 2017 is considered the reference time, while 18 October 2015 serves as the prediction time.

A uniform training set generation method is employed for all DL approaches. For the CIA and LGC datasets, MODIS–Landsat image pairs from different dates are randomly selected as reference images for each predicted time [22,26]. The images are cropped to a size of 256 × 256 with a stride of 200. For the Wuhan dataset, the reference–prediction time pairing method provided by the data provider is used. Given the relatively smaller dataset size, the cropping size is maintained at 256 × 256, but the stride is reduced to 125. This process yields 1260 training samples from the CIA dataset, 2464 training samples from the LGC dataset, and 637 training samples from the Wuhan dataset. During the testing phase, no cropping is performed, and the original image size is used as input.

STF-Trans, CTSTFM, and BCSR-STF all incorporate the self-attention mechanism from Transformers. In STF-Trans, the embedding dimension is set to 512, while CTSTFM uses an embedding dimension of 64. For BCSR-STF, due to its pyramid structure, the embedding dimensions are configured as (48, 96, 192, 192) across different levels. For BCSR-STF, the initial learning rate is 2 × 10⁻⁴. All deep learning-based methods are trained on one NVIDIA RTX 3090 GPU using the data enhancement strategy of image random flip or rotation and the training strategy of starting from scratch; no other tricks are used.

To comprehensively evaluate the performance of BCSR-STF, both quantitative and qualitative metrics are employed. (1) Quantitative metrics: Six evaluation metrics are used [26]: Root Mean Square Error (RMSE), Structure Similarity Index (SSIM), Universal Image Quality Index (UIQI), Correlation Coefficient (CC), Spectral Angle Mapper (SAM), and Erreur Relative Global Adimensionnelle de Synthène (ERGAS). Lower values of RMSE, SAM, and ERGAS, alongside higher values of SSIM, UIQI, and CC, signify a more reliable and accurate fusion outcome. These metrics comprehensively assess error, structural fidelity, spectral consistency, and overall synthesis quality of the fused images. (2) Qualitative metrics: To visualize the results, the NIR–Red–Green channels are used as RGB channels for image rendering, highlighting the vegetation-dominated scenes in the datasets. Additionally, the Average Absolute Difference (AAD) Map is employed to visually compare pixel-wise absolute error values between the predicted images and the ground truth. The error magnitude is represented using a color gradient, enabling an intuitive understanding of error distribution across the image.

3.3. Experimental Results for CIA

For the CIA dataset, the input data consists of a MODIS–Landsat data pair from 25 November 2001 and MODIS data from 12 January 2002. During this period, significant tonal differences exist between images from different time points due to phenological changes. Figure 5 presents the prediction results for the Landsat data from 12 January 2002 using various STF methods. Figure 5a depicts the true image. While all methods capture phenological changes to some extent, a detailed analysis reveals noticeable color distortions in the irrigation areas in the results of STARFM, FSDAF, and Fit-FC. These distortions are likely due to the single-band computation approach employed by traditional methods. The EDCSTFN model, which emphasizes temporal changes, effectively captures phenological changes during the observation period. However, it struggles with structural reconstruction, leading to a loss of spatial details and resulting in a blurry appearance.

Figure 6 presents the predicted results for the zoomed-in subregion within the black box in Figure 5a, along with the AAD map comparing them to the true data. It is observed that STARFM and FSDAF exhibit significant color spots, while the EDCSTFN results display excessively blurred boundaries, exceeding the acceptable range. In contrast, other DL-based methods provide more natural color representation but still exhibit some issues. For example, MLFF-GAN, which focuses on temporal changes, introduces unpleasant noise in its results. STF-Trans, which emphasizes spatial detail enhancement, has lower clarity, likely due to its global feature extraction approach neglecting local high-frequency information. CTSTFM demonstrates notable errors in capturing phenological changes in the upper-left corner. The AAD map indicates that the proposed method achieves the smallest error, highlighting its superior performance in the phenological change prediction task.

Table 1. Quantitative evaluation results with different methods on the CIA dataset.

	Band	STARFM	FSDAF	Fit-FC	EDCSTFN	GAN-STFM	MLFF-GAN	STF-Trans	CTSTFM	Proposed
RMSE	1	0.0166	0.0161	0.0157	0.0162	0.0126	0.0122	0.0123	0.0133	0.0118
	2	0.0250	0.0234	0.0235	0.0255	0.0180	0.0180	0.0167	0.0173	0.0164
	3	0.0399	0.0364	0.0376	0.0436	0.0293	0.0286	0.0267	0.0273	0.0261
	4	0.0496	0.0509	0.0472	0.0445	0.0401	0.0395	0.0374	0.0371	0.0370
	5	0.0460	0.0459	0.0468	0.0477	0.0396	0.0374	0.0362	0.0375	0.0348
	6	0.0379	0.0384	0.0385	0.0398	0.0344	0.0334	0.0324	0.0316	0.0302
	Avg	0.0358	0.0352	0.0349	0.0362	0.0290	0.0282	0.0270	0.0273	0.0260
SSIM	1	0.8928	0.9007	0.8939	0.8850	0.9250	0.9286	0.9306	0.9271	0.9359
	2	0.8464	0.8594	0.8536	0.8259	0.8966	0.8956	0.9030	0.9026	0.9084
	3	0.7739	0.7910	0.7873	0.7165	0.8413	0.8463	0.8511	0.8543	0.8594
	4	0.6811	0.6740	0.6785	0.7147	0.7608	0.7687	0.7710	0.7835	0.7873
	5	0.7429	0.7498	0.7462	0.7377	0.8006	0.8056	0.8096	0.8080	0.8160
	6	0.7748	0.7800	0.7723	0.7749	0.8178	0.8193	0.8284	0.8265	0.8318
	Avg	0.7853	0.7925	0.7886	0.7758	0.8404	0.8440	0.8489	0.8503	0.8565
UIQI	1	0.8140	0.8293	0.8190	0.8152	0.8979	0.9113	0.9201	0.9107	0.9245
	2	0.8152	0.8387	0.8275	0.8023	0.9107	0.9145	0.9279	0.9225	0.9312
	3	0.8165	0.8498	0.8400	0.7815	0.9132	0.9178	0.9294	0.9264	0.9336
	4	0.8275	0.8262	0.8370	0.8806	0.8976	0.9043	0.9128	0.9178	0.9184
	5	0.9222	0.9247	0.9224	0.9174	0.9444	0.9491	0.9551	0.9528	0.9576
	6	0.9206	0.9215	0.9193	0.9174	0.9368	0.9408	0.9477	0.9475	0.9521
	Avg	0.8527	0.8650	0.8609	0.8524	0.9168	0.9230	0.9322	0.9296	0.9363
CC	1	0.8320	0.8370	0.8401	0.8331	0.9022	0.9116	0.9230	0.9148	0.9265
	2	0.8369	0.8501	0.8473	0.8234	0.9153	0.9160	0.9292	0.9247	0.9320
	3	0.8454	0.8654	0.8551	0.8013	0.9165	0.9198	0.9306	0.9275	0.9345
	4	0.8344	0.8281	0.8459	0.8813	0.8994	0.9049	0.9141	0.9187	0.9186
	5	0.9222	0.9249	0.9238	0.9180	0.9451	0.9497	0.9554	0.9532	0.9577
	6	0.9210	0.9217	0.9198	0.9177	0.9376	0.9412	0.9484	0.9478	0.9523
	Avg	0.8653	0.8712	0.8720	0.8625	0.9194	0.9239	0.9334	0.9311	0.9369
ERGAS	ALL	1.3146	1.2666	1.2612	1.3488	1.0298	1.0047	0.9640	0.9931	0.9319
SAM	ALL	11.1256	10.9104	10.8326	11.4756	8.8898	8.6679	8.0970	8.2494	7.8974

The best values of the index are marked in bold.

lists the RMSE, SSIM, UIQI, CC, ERGAS, and SAM evaluation metrics for each method on the CIA dataset, with the best values highlighted in bold. The results demonstrate that the proposed method outperforms others across all metrics, showing significant advantages in radiometric accuracy, structural restoration, and spectral fidelity.

3.4. Experimental Results for LGC

For the LGC dataset, the input data includes MODIS–Landsat data pairs from 26 November 2004 and MODIS data from 12 December 2004. Significant differences exist between the two temporal scenes due to the impact of flooding. Compared to the prediction of phenological changes, predicting sudden land cover changes is more challenging. Figure 7 shows the predicted results for the LGC data on 12 December 2004 using different methods. Overall, except for CTSTFM, all other methods made relatively accurate predictions of the flood-affected areas. However, due to the significant spatial resolution difference between MODIS and Landsat, and the reliance on MODIS imagery at the prediction time for spatial information on the changes, the details of the flood-affected areas could not be perfectly restored. A zoomed-in view of the black-boxed region from the true Landsat image in Figure 7a is taken, and the AAD map for that region is calculated, as shown in Figure 8. It can be observed that the advantages of DL methods on the LGC dataset are not as pronounced as those on the CIA dataset. From the AAD results, FSDAF, despite being a traditional method based on simple theories, demonstrates relatively good fusion performance due to its consideration of both class changes and spatial reconstruction. In contrast, CTSTFM, which emphasizes spatial detail enhancement, fails to capture temporal changes, resulting in significant errors in the flood-affected areas. The proposed method continues to demonstrate the smallest error values.

Table 2. Quantitative evaluation results with different methods on the LGC dataset.

	Band	STARFM	FSDAF	Fit-FC	EDCSTFN	GAN-STFM	MLFF-GAN	STF-Trans	CTSTFM	Proposed
RMSE	1	0.0143	0.0149	0.0140	0.0151	0.0146	0.0161	0.0142	0.0167	0.0143
	2	0.0200	0.0207	0.0201	0.0200	0.0207	0.0223	0.0203	0.0234	0.0214
	3	0.0251	0.0258	0.0251	0.0257	0.0264	0.0269	0.0260	0.0320	0.0256
	4	0.0376	0.0397	0.0385	0.0394	0.041	0.0400	0.0360	0.0532	0.0351
	5	0.0568	0.0621	0.0565	0.0590	0.054	0.0533	0.0574	0.0660	0.0503
	6	0.0455	0.0515	0.0446	0.0407	0.0399	0.0404	0.0411	0.0476	0.0372
	Avg	0.0332	0.0358	0.0331	0.0333	0.0328	0.0332	0.0325	0.0398	0.0306
SSIM	1	0.9132	0.9125	0.9233	0.9228	0.9185	0.9059	0.9217	0.9166	0.9268
	2	0.8730	0.8709	0.8800	0.8897	0.8801	0.8709	0.8856	0.8843	0.8888
	3	0.8350	0.8331	0.8438	0.8455	0.8413	0.8356	0.8498	0.8391	0.8562
	4	0.7292	0.7294	0.7405	0.7083	0.7037	0.7239	0.7409	0.6910	0.7593
	5	0.5697	0.5220	0.5532	0.5513	0.5696	0.5948	0.5797	0.5766	0.6241
	6	0.6408	0.5754	0.6274	0.6267	0.6438	0.6533	0.6481	0.6473	0.6895
	Avg	0.7601	0.7405	0.7614	0.7574	0.7595	0.7641	0.7710	0.7591	0.7908
UIQI	1	0.7152	0.7062	0.7124	0.6132	0.6844	0.6839	0.7313	0.5813	0.7354
	2	0.7019	0.6890	0.6943	0.6560	0.6807	0.7125	0.7283	0.5637	0.7262
	3	0.7072	0.6965	0.7007	0.6573	0.696	0.7179	0.7184	0.5366	0.7541
	4	0.7857	0.7794	0.7827	0.7431	0.766	0.7826	0.8193	0.6991	0.8370
	5	0.7531	0.7198	0.7313	0.6645	0.7659	0.7940	0.7663	0.7436	0.8169
	6	0.7205	0.6531	0.7021	0.7316	0.7798	0.7918	0.7849	0.7713	0.8215
	Avg	0.7306	0.7073	0.7206	0.6776	0.7288	0.7471	0.7581	0.6493	0.7818
CC	1	0.7158	0.7076	0.7160	0.6582	0.6878	0.6870	0.7322	0.6073	0.7363
	2	0.7071	0.6916	0.7007	0.6958	0.6888	0.7169	0.7283	0.5902	0.7293
	3	0.7130	0.7004	0.7086	0.6834	0.6994	0.7192	0.7188	0.5469	0.7564
	4	0.8075	0.8011	0.8098	0.7832	0.769	0.7871	0.8300	0.7243	0.8375
	5	0.7909	0.7666	0.7832	0.7695	0.7864	0.7993	0.8109	0.7750	0.8282
	6	0.7873	0.7473	0.7786	0.7871	0.7883	0.7977	0.8153	0.7949	0.8293
	Avg	0.7536	0.7358	0.7495	0.7295	0.7366	0.7512	0.7726	0.6731	0.7862
ERGAS	ALL	2.0655	2.2230	2.0322	2.0245	1.9837	2.0300	1.9938	2.3816	1.8816
SAM	ALL	16.2826	17.0293	16.2303	16.8170	16.7738	16.5577	15.9620	18.0406	15.3922

The best values of the index are marked in bold.

summarizes the performance metrics for each method applied to the LGC dataset, with the best values highlighted in bold. The results clearly demonstrate that the proposed method achieves optimal performance across most metrics. Notably, in the last three bands, the proposed method significantly outperforms the others, indicating its superior performance even in scenarios involving substantial changes.

3.5. Experimental Results for Wuhan

Compared to the CIA and LGC datasets, the Wuhan dataset exhibits three distinct characteristics: first, it covers an urban area with more complex land cover textures; second, it comprises Landsat–GF images rather than the traditional MODIS–Landsat pairs; and third, the dataset is relatively small in scale. In this study, 30 October 2017 is selected as the reference time and 18 October 2015 as the prediction time. Figure 9 shows the global fusion results of different STF models on the Wuhan dataset. Compared to Figure 9a, the results of CTSTFM, which focuses on spatial detail reconstruction, exhibit noticeable color errors relative to the true image. According to

Table 3. Quantitative evaluation results with different methods on the Wuhan dataset.

	Band	STARFM	FSDAF	Fit-FC	EDCSTFN	GAN-STFM	MLFF-GAN	STF-Trans	CTSTFM	Proposed
RMSE	1	0.0583	0.0574	0.0636	0.0307	0.0257	0.0485	0.0224	0.0223	0.0195
	2	0.0495	0.0478	0.0522	0.0294	0.0341	0.0535	0.0294	0.0288	0.0246
	3	0.0459	0.0495	0.0437	0.0334	0.0279	0.0505	0.0363	0.0387	0.0274
	4	0.0583	0.0599	0.0515	0.0507	0.0635	0.0568	0.0537	0.0970	0.0413
	Avg	0.0530	0.0536	0.0528	0.0361	0.0378	0.0523	0.0354	0.0467	0.0282
SSIM	1	0.6448	0.6396	0.6239	0.8070	0.8433	0.6907	0.8405	0.8634	0.8820
	2	0.6662	0.6437	0.6620	0.8128	0.8068	0.6644	0.7807	0.8093	0.8470
	3	0.6838	0.6171	0.6911	0.8029	0.8497	0.6513	0.7604	0.7933	0.8483
	4	0.5599	0.5214	0.5844	0.7172	0.7146	0.5634	0.6079	0.3777	0.7207
	Avg	0.6387	0.6054	0.6403	0.7850	0.8036	0.6424	0.7474	0.7109	0.8245
UIQI	1	0.6375	0.5972	0.5536	0.7658	0.8159	0.6103	0.7599	0.7275	0.8579
	2	0.7041	0.6336	0.6407	0.8133	0.8183	0.6268	0.7692	0.7562	0.8643
	3	0.7969	0.7120	0.7690	0.8464	0.8965	0.7019	0.7994	0.7819	0.9106
	4	0.7769	0.7405	0.7800	0.8600	0.8460	0.7586	0.8081	0.1907	0.8802
	Avg	0.7289	0.6708	0.6858	0.8214	0.8442	0.6744	0.7842	0.6141	0.8782
CC	1	0.8106	0.7868	0.8519	0.9129	0.8945	0.7761	0.8572	0.8264	0.9013
	2	0.7930	0.7599	0.8531	0.9139	0.8994	0.7645	0.8655	0.8611	0.9007
	3	0.8636	0.7884	0.8845	0.9388	0.9325	0.7874	0.8926	0.8780	0.9235
	4	0.8104	0.7810	0.8430	0.9077	0.9033	0.7863	0.7842	0.3400	0.8894
	Avg	0.8194	0.7790	0.8581	0.9183	0.9074	0.7786	0.8699	0.7264	0.9077
ERGAS	ALL	3.7344	3.7053	3.9652	2.1718	2.0695	3.4732	1.9407	2.1609	1.5933
SAM	ALL	17.7869	19.3630	18.8012	14.5189	13.4744	19.3714	15.2835	17.1857	12.4134

The best values of the index are marked in bold.

, this issue likely stems from poor prediction in the near-infrared band, indicating insufficient learning of near-infrared features by the model. FSDAF shows noticeable artifacts in the densely built-up areas near the center of the image. For the water region at the top of the result image, MLFF-GAN, STF-Trans, and CTSTFM all display visible stitching marks. Special attention is given to the water area marked by the black box in Figure 9a, where significant changes occurred during the study period. Compared to 18 October 2015, the water area decreased by 30 October 2017. The zoomed-in view and corresponding error map are shown in Figure 10. The AAD map reveals that the three traditional methods and the temporal change-focused MLFF-GAN failed to capture this change effectively. In contrast, the proposed method is more sensitive to changes in the water boundary than the spatial detail enhancement-focused STF-Trans and CTSTFM. Furthermore, it demonstrates superior overall detail reconstruction compared to the temporal change-focused EDCSTFN.

Table 3 presents a quantitative comparison of fusion results on the Wuhan dataset. EDCSTFN achieves the highest CC metric; however, its performance on other metrics is less competitive, indicating that although EDCSTFN-generated images align well with the overall trends of the real images, they contain relatively large local errors. In contrast, the proposed method attains the second-highest CC value and outperforms all other methods on the remaining metrics.

4. Discussion

4.1. Ablation Studies

In this section, we perform ablation studies on the CIA, LGC, and Wuhan datasets.

4.1.1. Progressive Spatiotemporal Feature Fusion and Restoration

In the proposed BCSR-STF, the multi-layer PSTFR module serves as the core component, designed to perform iterative processing across multiple scales. To evaluate the effectiveness of this multi-layer structure, a single-layer variant named BCSR-STF-S is introduced for an ablation study. In BCSR-STF-S, feature extraction is applied only once to the input images from both the reference and prediction times, while all other settings remain unchanged. The results, presented in Figure 11, demonstrate that BCSR-STF consistently outperforms BCSR-STF-S. This improvement is attributed to the multi-layer design, which provides greater adaptability and better addresses challenges arising from varying image resolutions and temporal differences. At coarser scales, the model captures global information using lower-resolution features, whereas at finer scales, it restores high-resolution details and enhances spatiotemporal features, thereby mitigating the risk of information loss.

4.1.2. Bidirectional Cross Fusion

The BCF module significantly enhances spatiotemporal fusion by simultaneously addressing both the time and scale directions. To evaluate the effectiveness of this Bidirectional Cross Fusion module, two ablation experiments were conducted by removing the fusion modules for the temporal and scale directions. Specifically, the model without temporal fusion was designated BCSR-STF-NT (No Time direction), and the model without scale fusion was designated BCSR-STF-NS (No Scale direction).

For BCSR-STF-NT, we removed the cross-temporal feature fusion in the time direction and only performed spatiotemporal fusion in the scale direction, with other settings unchanged. Similarly, BCSR-STF-NS removed the cross-scale feature fusion in the scale direction and only performed spatiotemporal fusion in the time direction. The results of the ablation experiments, shown in Figure 12, demonstrate that BCSR-STF outperforms both BCSR-STF-NT and BCSR-STF-NS. This improvement can be attributed to the design of the Bidirectional Cross Fusion module. The fusion in the time direction helps capture temporal variation features, while the fusion in the scale direction enhances the detailed information between different resolutions. By simultaneously utilizing both directions of fusion, BCSR-STF can better handle spatiotemporal differences, providing more comprehensive and detailed spatiotemporal features.

4.1.3. Global Spectral Restoration and Feature Enhancement

The GSRFE module significantly enhances spatiotemporal fusion by adaptively adjusting spectral features. To validate the effectiveness of the GSRFE module, we designed two ablation experiments: one without spectral adjustment and one with only spectral adjustment but no feature enhancement. Specifically, we named the model without spectral adjustment as BCSR-STF-NA (No Spectral Adjustment) and the model with only spectral adjustment but no feature enhancement as BCSR-STF-OA (Only Spectral Adjustment). In BCSR-STF-NA, we removed the GSRFE module and directly used the spatiotemporal fusion result $F_{st}^{\left(i\right)}$ as the output. For BCSR-STF-OA, we only used the result $F_{st}^{\left(i\right)″}$ that had undergone spectral adaptive adjustment without any feature enhancement. Other settings remained unchanged.

The results of the ablation experiments, shown in Figure 13, indicate that BCSR-STF outperforms both BCSR-STF-NA and BCSR-STF-OA. This improvement can be attributed to the combination of spectral adaptive adjustment and feature enhancement in the GSRFE module. The spectral adaptive adjustment helps correct spectral inconsistencies caused by sensor errors, while feature enhancement further improves the expressive capability of the spatiotemporal fusion features.

4.2. Computation Load

In this study, FLOPs (Floating-Point Operations per Second) and the number of parameters are used to measure the computational cost of the models. FLOPs represent the total number of floating-point operations required during the inference process and serve as a key metric for evaluating computational complexity. The input data size is set to (1, 6, 256, 256), where 1 denotes the batch size, 6 represents the number of input channels, and 256 × 256 corresponds to the resolution of the input image. By calculating FLOPs and the number of parameters, the computational efficiency and resource requirements of the models are comprehensively assessed. The results are presented in

Table 4. Model efficiency analysis of six deep learning-based spatiotemporal fusion models.

MODEL		Param. (M)	FLOPs
EDCSTFN		0.28	1.86 × 1010
GAN-STFM	Generator	0.58	3.78 × 1010
	Discriminator	3.67	1.03 × 107
MLFF-GAN	Generator	5.93	1.36 × 1010
	Discriminator	2.78	3.77 × 109
STF-Trans		23.34	1.74 × 1011
CTSTFM		6.30	3.8 × 1011
BCSR-STF		34.80	2.71 × 1010

.

As shown in Table 4, although the parameter count of BCSR-STF reaches 34.80 M, which is higher than that of other lightweight models, its FLOPs are only 2.71 × 10¹⁰, significantly lower than the self-attention-based models STF-Trans and CTSTFM. This advantage is primarily attributed to its pyramid structure and windowed attention mechanism, which strike a well-balanced trade-off between computational efficiency and model performance. Furthermore, the results from three experimental datasets demonstrate that BCSR-STF exhibits the best robustness, effectively adapting to diverse scenarios and varying data distributions. For dense prediction tasks such as image fusion, a moderate increase in parameter count is generally accepted in exchange for improved model expressiveness and performance.

5. Conclusions

This work proposes BCSR-STF, a model designed to generate time series remote sensing data via spatiotemporal fusion. BCSR-STF integrates a multi-layer PSTFR structure, enabling iterative optimization of spatiotemporal features across multiple scales and enhancing adaptability to variations in resolution and temporal differences. The core components of PSTFR are the Bidirectional Cross Fusion (BCF) and Global Spectral Restoration and Feature Enhancement (GSRFE) modules. BCF significantly improves the fusion of spatiotemporal features by simultaneously processing information along both temporal and scale dimensions. GSRFE effectively corrects sensor errors and spectral inconsistencies through adaptive spectral adjustment and feature enhancement. The effectiveness of each module is validated through a series of ablation experiments. Comparative studies demonstrate the superior performance of BCSR-STF. However, the model demands substantial computational resources, potentially limiting its applicability in resource-constrained environments. Future work will focus on reducing computational complexity and resource consumption through algorithmic optimizations, such as model pruning and quantization, as well as the design of lightweight network architectures.