Search Results (114)

Pan-sharpening aims to generate high-resolution multispectral (HRMS) images by combining high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) data, while maintaining the symmetry of spatial and spectral characteristics. Traditional convolutional neural networks (CNNs) struggle with global dependency modeling due to local receptive fields, and Transformer-based models are computationally expensive. Recent Mamba models offer linear complexity and effective global modeling. However, existing Mamba-based methods lack sensitivity to local feature variations, leading to suboptimal fine-detail preservation. To address this, we propose a Conditional Skipping Mamba Network (CSMN), which enhances global-local feature fusion symmetrically through two modules: (1) the Adaptive Mamba Module (AMM), which improves global perception using adaptive spatial-frequency integration; and (2) the Cross-domain Mamba Module (CDMM), optimizing cross-domain spectral-spatial representation. Experimental results on the IKONOS and WorldView-2 datasets demonstrate that CSMN surpasses existing state-of-the-art methods in achieving superior spectral consistency and preserving spatial details, with performance that is more symmetric in fine-detail preservation. Full article

Land surface temperature (LST) plays a pivotal role in many environmental sectors. Unfortunately, thermal bands produced by instruments that are onboard satellites have limited spatial resolutions; this seriously impairs their potential usefulness. In this study, we propose an automatic procedure for the spatial downscaling of the two 100 m thermal infrared (TIR) bands of LandSat 8/9, captured by the TIR spectrometer (TIRS), by exploiting the bands of the optical instrument. The problem of fusion of heterogeneous data is approached as hypersharpening: each of the two sharpening images is synthesized following data assimilation concepts, with the linear combination of 30 m optical bands and the 15 m panchromatic (Pan) image that maximizes the correlation with each thermal channel at its native 100 m scale. The TIR bands resampled at 15 m are sharpened, each by its own synthetic Pan. On two different scenes of an OLI-TIRS image, the proposed approach is compared with 100 m to 15 m pansharpening, carried out uniquely by means of the Pan image of OLI and with the two high-resolution assimilated thermal images that are used for hypersharpening the two TIRS bands. Besides visual evaluations of the temperature maps, statistical indexes measuring radiometric and spatial consistencies are provided and discussed. The superiority of the proposed approach is highlighted: the classical pansharpening approach is radiometrically accurate but weak in the consistency of spatial enhancement. Conversely, the assimilated TIR bands, though adequately sharp, lose more than 20% of radiometric consistency. Our proposal trades off the benefits of its counterparts in a unique method. Full article

Pan-sharpening aims to generate high-resolution (HR) multispectral (MS) images by fusing HR panchromatic (PAN) and low-resolution (LR) MS images covering the same area. However, due to the lack of real HR MS reference images, how to accurately evaluate the quality of a fused image without reference is challenging. On the one hand, most methods evaluate the quality of the fused image using the full-reference indices based on the simulated experimental data on the popular Wald’s protocol; however, this remains controversial to the full-resolution data fusion. On the other hand, existing limited no reference methods, most of which depend on manually crafted features, cannot fully capture the sensitive spatial/spectral distortions of the fused image. Therefore, this paper proposes a perceptual quality assessment method based on deep feature similarity measure. The proposed network includes spatial/spectral feature extraction and similarity measure (FESM) branch and overall evaluation network. The Siamese FESM branch extracts the spatial and spectral deep features and calculates the similarity of the corresponding pair of deep features to obtain the spatial and spectral feature parameters, and then, the overall evaluation network realizes the overall quality assessment. Moreover, we propose to quantify both the overall precision of all the training samples and the variations among different fusion methods in a batch, thereby enhancing the network’s accuracy and robustness. The proposed method was trained and tested on a large subjective evaluation dataset comprising 13,620 fused images. The experimental results suggested the effectiveness and the competitive performance. Full article

The objective of pan-sharpening is to effectively fuse high-resolution panchromatic (PAN) images with limited spectral information and low-resolution multispectral (LR-MS) images, thereby generating a fused image with a high spatial resolution and rich spectral information. However, current fusion techniques face significant challenges, including insufficient edge detail, spectral distortion, increased noise, and limited robustness. To address these challenges, we propose a multi-frequency spectral–spatial interaction enhancement network (MFSINet) that comprises the spectral–spatial interactive fusion (SSIF) and multi-frequency feature enhancement (MFFE) subnetworks. The SSIF enhances both spatial and spectral fusion features by optimizing the characteristics of each spectral band through band-aware processing. The MFFE employs a variant of wavelet transform to perform multiresolution analyses on remote sensing scenes, enhancing the spatial resolution, spectral fidelity, and the texture and structural features of the fused images by optimizing directional and spatial properties. Moreover, qualitative analysis and quantitative comparative experiments using the IKONOS and WorldView-2 datasets indicate that this method significantly improves the fidelity and accuracy of the fused images. Full article

Image preprocessing and fusion are commonly used for enhancing remote-sensing images, but the resulting images often lack useful spatial features. As the majority of research on image fusion has concentrated on the satellite domain, the image-fusion task for Unmanned Aerial Vehicle (UAV) images has received minimal attention. This study investigated an image-improvement strategy by integrating image preprocessing and fusion tasks for UAV images. The goal is to improve spatial details and avoid color distortion in fused images. Techniques such as image denoising, sharpening, and Contrast Limited Adaptive Histogram Equalization (CLAHE) were used in the preprocessing step. The unsharp mask algorithm was used for image sharpening. Wiener and total variation denoising methods were used for image denoising. The image-fusion process was conducted in two steps: (1) fusing the spectral bands into one multispectral image and (2) pansharpening the panchromatic and multispectral images using the PanColorGAN model. The effectiveness of the proposed approach was evaluated using quantitative and qualitative assessment techniques, including no-reference image quality assessment (NR-IQA) metrics. In this experiment, the unsharp mask algorithm noticeably improved the spatial details of the pansharpened images. No preprocessing algorithm dramatically improved the color quality of the enhanced images. The proposed fusion approach improved the images without importing unnecessary blurring and color distortion issues. Full article

Beijing Satellite 3 is a high-performance optical remote sensing satellite with a spatial resolution of 0.3–0.5 m. It can provide timely and independent ultra-high-resolution spatial big data and comprehensive spatial information application services. At present, there is no relevant research on the fusion method of BJ-3A satellite images. In many applications, high-resolution panchromatic images alone are insufficient. Therefore, it is necessary to fuse them with multispectral images that contain spectral color information. Currently, there is a lack of research on the fusion method of BJ-3A satellite images. This article explores six traditional pixel-level fusion methods (HPF, HCS, wavelet, modified-IHS, PC, and Brovey) for fusing the panchromatic image and multispectral image of the BJ-3A satellite. The fusion results were analyzed qualitatively from two aspects: spatial detail enhancement capability and spectral fidelity. Five indicators, namely mean, standard deviation, entropy, correlation coefficient, and average gradient, were used for quantitative analysis. Finally, the fusion results were comprehensively evaluated from three aspects: spectral curves of ground objects, absolute error figure, and object-oriented classification effects. The findings of the research suggest that the fusion method known as HPF is the optimum and appropriate technique for fusing panchromatic and multispectral images obtained from BJ-3A. These results can be utilized as a guide for the implementation of BJ-3A panchromatic and multispectral data fusion in real-world scenarios. Full article

Pan-sharpening is a fusion process that combines a low-spatial resolution, multi-spectral image that has rich spectral characteristics with a high-spatial resolution panchromatic (PAN) image that lacks spectral characteristics. Most previous learning-based approaches rely on the scale-shift assumption, which may not be applicable in the full-resolution domain. To solve this issue, we regard pan-sharpening as a multi-task problem and propose a Siamese network with Gradient-based Spatial Attention (GSA-SiamNet). GSA-SiamNet consists of four modules: a two-stream feature extraction module, a feature fusion module, a gradient-based spatial attention (GSA) module, and a progressive up-sampling module. In the GSA module, we use Laplacian and Sobel operators to extract gradient information from PAN images. Spatial attention factors, learned from the gradient prior, are multiplied during the feature fusion, up-sampling, and reconstruction stages. These factors help to keep high-frequency information on the feature map as well as suppress redundant information. We also design a multi-resolution loss function that guides the training process under the constraints of both reduced- and full-resolution domains. The experimental results on WorldView-3 satellite images obtained in Moscow and San Juan demonstrate that our proposed GSA-SiamNet is superior to traditional and other deep learning-based methods. Full article

Unmanned aerial vehicles (UAVs) are extensively used to gather remote sensing data, offering high image resolution and swift data acquisition despite being labor-intensive. In contrast, satellite-based remote sensing, providing sub-meter spatial resolution and frequent revisit times, could serve as an alternative data source for phenotyping. In this study, we separately evaluated pan-sharpened Pléiades satellite imagery (50 cm) and UAV imagery (2.5 cm) to phenotype durum wheat in small-plot (12 m × 1.10 m) breeding trials. The Gaussian process regression (GPR) algorithm, which provides predictions with uncertainty estimates, was trained with spectral bands and а selected set of vegetation indexes (VIs) as independent variables. Grain protein content (GPC) was better predicted with Pléiades data at the growth stage of 20% of inflorescence emerged but with only moderate accuracy (validation R²: 0.58). The grain yield (GY) and protein yield (PY) were better predicted using UAV data at the late milk and watery ripe growth stages, respectively (validation: R² 0.67 and 0.62, respectively). The cumulative VIs (the sum of VIs over the available images within the growing season) did not increase the accuracy of the models for either sensor. When mapping the estimated parameters, the spatial resolution of Pléiades revealed certain limitations. Nevertheless, our findings regarding GPC suggested that the usefulness of pan-sharpened Pléiades images for phenotyping should not be dismissed and warrants further exploration, particularly for breeding experiments with larger plot sizes. Full article

Achieving a balance between spectral resolution and spatial resolution in multi-spectral remote sensing images is challenging due to physical constraints. Consequently, pan-sharpening technology was developed to address this challenge. While significant progress was recently achieved in deep-learning-based pan-sharpening techniques, most existing deep learning approaches face two primary limitations: (1) convolutional neural networks (CNNs) struggle with long-range dependency issues, and (2) significant detail loss during deep network training. Moreover, despite these methods’ pan-sharpening capabilities, their generalization to full-sized raw images remains problematic due to scaling disparities, rendering them less practical. To tackle these issues, we introduce in this study a multi-spectral remote sensing image fusion network, termed TAMINet, which leverages a two-stream coordinate attention mechanism and multi-detail injection. Initially, a two-stream feature extractor augmented with the coordinate attention (CA) block is employed to derive modal-specific features from low-resolution multi-spectral (LRMS) images and panchromatic (PAN) images. This is followed by feature-domain fusion and pan-sharpening image reconstruction. Crucially, a multi-detail injection approach is incorporated during fusion and reconstruction, ensuring the reintroduction of details lost earlier in the process, which minimizes high-frequency detail loss. Finally, a novel hybrid loss function is proposed that incorporates spatial loss, spectral loss, and an additional loss component to enhance performance. The proposed methodology’s effectiveness was validated through experiments on WorldView-2 satellite images, IKONOS, and QuickBird, benchmarked against current state-of-the-art techniques. Experimental findings reveal that TAMINet significantly elevates the pan-sharpening performance for large-scale images, underscoring its potential to enhance multi-spectral remote sensing image quality. Full article

Pan-sharpening is an important means to improve the spatial resolution of multispectral (MS) images. Although a large number of pan-sharpening methods have been developed, improving the spatial resolution of MS while effectively maintaining its spectral information has not been well solved so far, and it has also been taken as a criterion to measure whether the sharpened product can meet the practical needs. The back-projection (BP) method iteratively injects spectral information backwards into the sharpened results in a post-processing manner, which can effectively improve the generally unsatisfied spectral consistency problem in pan-sharpening methods. Although BP has received some attention in recent years in pan-sharpening research, the existing related work is basically limited to the direct utilization of the BP process and lacks a more in-depth intrinsic integration with pan-sharpening. In this paper, we analyze the current problems of improving the spectral consistency based on BP in pan-sharpening, and the main innovative works carried out on this basis include the following: (1) We introduce the spatial consistency condition and propose the spatial–spectral BP (SSBP) method, which takes into account both spatial and spectral consistency conditions, to improve the spectral quality while effectively solving the problem of spatial distortion in the results. (2) The proposed SSBP method is analyzed theoretically, and the convergence condition of SSBP and a more relaxed convergence condition for a specific BP type, degradation transpose BP, are given and proved theoretically. (3) Fast computation of BP and SSBP is investigated, and non-iterative fast BP (FBP) and fast SSBP algorithms (FSSBP) methods are given in a closed-form solution with significant improvement in computational efficiency. Experimental comparisons with combinations formed by seven different BP-related post-processing methods and up to 18 typical base methods show that the proposed methods are generally applicable to the optimization of the spatial–spectral quality of various sharpening methods. The fast method improves the computational speed by at least 27.5 times compared to the iterative version while maintaining the evaluation metrics well. Full article

Salt marshes are highly important wetlands; however, external pressures are causing their widespread deterioration and loss. Continuous monitoring of their extent is paramount for the preservation and recovery of deteriorated and threatened salt marshes. In general, moderate-resolution satellite remote sensing data allow for the accurate detection of salt marsh shorelines; however, their detection in narrow and fringing salt marshes remains challenging. This study aims to evaluate the ability of Landsat-5 (TM), Landsat-7 (ETM+), and Sentinel-2 (MSI) data to be used to accurately determine the shoreline of narrow and fringing salt marshes, focusing on three regions of the Aveiro lagoon in Mira, Ílhavo and S. Jacinto channels. Shorelines were determined considering the Normalized Difference Vegetation Index (NDVI), and the accuracy of this methodology was evaluated against reference shorelines by computing the Root Mean Square Error (RMSE). Once validated, the method was used to determine historical salt marsh shorelines, and rates of change between 1984 and 2022 were quantified and analyzed in the three locations. Results evidence that the 30 m resolution Landsat data accurately describe the salt marsh shoreline (RMSE~15 m) and that the accuracy is maintained when increasing the spatial resolution through pan-sharpening or when using 10 m resolution Sentinel-2 (MSI) data. These also show that the salt marshes of the Ílhavo and S. Jacinto channels evolved similarly, with salt marsh shoreline stability before 2000 followed by retreats after this year. At the end of the four decades of study, an average retreat of 66.23 ± 1.03 m and 46.62 ± 0.83 m was found, respectively. In contrast to these salt marshes and to the expected evolution, the salt marsh of the Mira Channel showed retreats before 2000, followed by similar progressions after this year, resulting in an average 2.33 ± 1.18 m advance until 2022. Full article

Pan-sharpening aims to create high-resolution spectrum images by fusing low-resolution hyperspectral (HS) images with high-resolution panchromatic (PAN) images. Inspired by the Swin transformer used in image classification tasks, this research constructs a three-stream pan-sharpening network based on the Swin transformer and a multi-scale feature extraction module. Unlike the traditional convolutional neural network (CNN) pan-sharpening model, we use the Swin transformer to establish global connections with the image and combine it with a multi-scale feature extraction module to extract local features of different sizes. The model combines the advantages of the Swin transformer and CNN, enabling fused images to maintain good local detail and global linkage by mitigating distortion in hyperspectral images. In order to verify the effectiveness of the method, this paper evaluates fused images with subjective visual and quantitative indicators. Experimental results show that the method proposed in this paper can better preserve the spatial and spectral information of images compared to the classical and latest models. Full article

Rapid and accurate estimation of forest aboveground biomass (AGB) with fine details is crucial for effective forest monitoring and management, where forest height plays a key role in AGB quantification. In this study, we propose a random forest (RF)-based down-scaling method to map forest height and biomass at a 15-m resolution by integrating Landsat 8 OLI and Ice, Cloud and Land Elevation Satellite-2 (ICESat-2) LiDAR data. ICESat-2 photon data are used to derive canopy parameters along 15-m segments, which are considered sample plots for the extrapolation of discrete forest height. Fourteen variables associated with spectral features, textual features and vegetation index are extracted from pan-sharpened Landsat 8 images. A regression function is established between these variables and ICESat-2-derived forest height to produce a 15-m continuous forest height distribution data based on the 30-m forest height product using the RF algorithm. Finally, a wall-to-wall forest AGB at 15-m spatial resolution is achieved by using an allometric model specific to the forest type and height. The Jilin Province in northeast China is taken as the study area, and the forest AGB estimation results reveal a density of 61.15 Mg/ha with a standard deviation of 89.46 Mg/ha. The $R^2$ between our predicted forest heights and the ICESat-2-derived heights reaches 0.93. Validation results at the county scale demonstrate reasonable correspondence between the estimated AGB and reference data, with consistently high $R^2$ value exceeding 0.65. This downscaling method provides a promising scheme to estimate spatial forest AGB with fine details and to enhance the accuracy of AGB estimation, which may facilitate carbon stock measurement and carbon cycle studies. Full article

Based on deep learning, various pan-sharpening models have achieved excellent results. However, most of them adopt simple addition or concatenation operations to merge the information of low spatial resolution multi-spectral (LRMS) images and panchromatic (PAN) images, which may cause a loss of detailed information. To tackle this issue, inspired by capsule networks, we propose a plug-and-play layer named modified dynamic routing layer (MDRL), which modifies the information transmission mode of capsules to effectively fuse LRMS images and PAN images. Concretely, the lower-level capsules are generated by applying transform operation to the features of LRMS images and PAN images, which preserve the spatial location information. Then, the dynamic routing algorithm is modified to adaptively select the lower-level capsules to generate the higher-level capsule features to represent the fusion of LRMS images and PAN images, which can effectively avoid the loss of detailed information. In addition, the previous addition and concatenation operations are illustrated as special cases of our MDRL. Based on MIPSM with addition operations and DRPNN with concatenation operations, two modified dynamic routing models named MDR–MIPSM and MDR–DRPNN are further proposed for pan-sharpening. Extensive experimental results demonstrate that the proposed method can achieve remarkable spectral and spatial quality. Full article

In recent years, the demand for very high geometric resolution satellite images has increased significantly. The pan-sharpening techniques, which are part of the data fusion techniques, enable the increase in the geometric resolution of multispectral images using panchromatic imagery of the same scene. However, it is not trivial to choose a suitable pan-sharpening algorithm: there are several, but none of these is universally recognized as the best for any type of sensor, in addition to the fact that they can provide different results with regard to the investigated scene. This article focuses on the latter aspect: analyzing pan-sharpening algorithms in relation to different land covers. A dataset of GeoEye-1 images is selected from which four study areas (frames) are extracted: one natural, one rural, one urban and one semi-urban. The type of study area is determined considering the quantity of vegetation included in it based on the normalized difference vegetation index (NDVI). Nine pan-sharpening methods are applied to each frame and the resulting pan-sharpened images are compared by means of spectral and spatial quality indicators. Multicriteria analysis permits to define the best performing method related to each specific area as well as the most suitable one, considering the co-presence of different land covers in the analyzed scene. Brovey transformation fast supplies the best results among the methods analyzed in this study. Full article