Next Article in Journal
Unsupervised Nonlinear Hyperspectral Unmixing Based on Bilinear Mixture Models via Geometric Projection and Constrained Nonnegative Matrix Factorization
Next Article in Special Issue
An Enhanced Linear Spatio-Temporal Fusion Method for Blending Landsat and MODIS Data to Synthesize Landsat-Like Imagery
Previous Article in Journal
Evolution and Controls of Large Glacial Lakes in the Nepal Himalaya
Previous Article in Special Issue
A Modified Spatiotemporal Fusion Algorithm Using Phenological Information for Predicting Reflectance of Paddy Rice in Southern China
Open AccessArticle

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

1
School of Automation, Northwestern Polytechnical University, Xi’an 710072, China
2
Department of Electronics and Informatics, Vrije Universiteit Brussel, 1050 Brussel, Belgium
*
Author to whom correspondence should be addressed.
Remote Sens. 2018, 10(5), 800; https://doi.org/10.3390/rs10050800
Received: 5 May 2018 / Revised: 14 May 2018 / Accepted: 14 May 2018 / Published: 21 May 2018
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods. View Full-Text
Keywords: convolutional neural network; deep learning; hyperspectral; multispectral; fusion convolutional neural network; deep learning; hyperspectral; multispectral; fusion
Show Figures

Graphical abstract

MDPI and ACS Style

Yang, J.; Zhao, Y.-Q.; Chan, J.C.-W. Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network. Remote Sens. 2018, 10, 800.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop