MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

Wang, Wenqing; Jia, Fei; Yang, Yifei; Mu, Kunpeng; Liu, Han

doi:10.3390/rs16193583

Open AccessArticle

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

by

Wenqing Wang

^1,2

,

Fei Jia

¹,

Yifei Yang

¹,

Kunpeng Mu

¹ and

Han Liu

^1,2,*

¹

School of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, China

²

Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi’an University of Technology, Xi’an 710048, China

^*

Author to whom correspondence should be addressed.

Remote Sens. 2024, 16(19), 3583; https://doi.org/10.3390/rs16193583

Submission received: 16 August 2024 / Revised: 12 September 2024 / Accepted: 23 September 2024 / Published: 26 September 2024

(This article belongs to the Special Issue Image Fusion and Object Detection Using Multi-Modal Remote Sensing Data)

Download

Browse Figures

Versions Notes

Abstract

Pansharpening refers to enhancing the spatial resolution of multispectral images through panchromatic images while preserving their spectral features. However, existing traditional methods or deep learning methods always have certain distortions in the spatial or spectral dimensions. This paper proposes a remote sensing spatial–spectral fusion method based on a multi-scale dual-stream convolutional neural network, which includes feature extraction, feature fusion, and image reconstruction modules for each scale. In terms of feature fusion, we propose a multi cascade module to better fuse image features. We also design a new loss function aim at enhancing the high degree of consistency between fused images and reference images in terms of spatial details and spectral information. To validate its effectiveness, we conduct thorough experimental analyses on two widely used remote sensing datasets: GeoEye-1 and Ikonos. Compared with the nine leading pansharpening techniques, the proposed method demonstrates superior performance in multiple key evaluation metrics.

Keywords: pansharpening; multispectral image; convolutional neural network; multiple cascaded modules

Share and Cite

MDPI and ACS Style

Wang, W.; Jia, F.; Yang, Y.; Mu, K.; Liu, H. MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network. Remote Sens. 2024, 16, 3583. https://doi.org/10.3390/rs16193583

AMA Style

Wang W, Jia F, Yang Y, Mu K, Liu H. MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network. Remote Sensing. 2024; 16(19):3583. https://doi.org/10.3390/rs16193583

Chicago/Turabian Style

Wang, Wenqing, Fei Jia, Yifei Yang, Kunpeng Mu, and Han Liu. 2024. "MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network" Remote Sensing 16, no. 19: 3583. https://doi.org/10.3390/rs16193583

APA Style

Wang, W., Jia, F., Yang, Y., Mu, K., & Liu, H. (2024). MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network. Remote Sensing, 16(19), 3583. https://doi.org/10.3390/rs16193583

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI