Next Article in Journal
Invasive Plant Species Establishment and Range Dynamics in Sri Lanka under Climate Change
Previous Article in Journal
MapEff: An Effective Graph Isomorphism Agorithm Based on the Discrete-Time Quantum Walk
Article Menu

Export Article

Open AccessArticle

A New Deep Learning Based Multi-Spectral Image Fusion Method

Department of Electrical Engineering, Hanyang University, Ansan 15588, Korea
Authors to whom correspondence should be addressed.
Entropy 2019, 21(6), 570;
Received: 12 May 2019 / Revised: 2 June 2019 / Accepted: 3 June 2019 / Published: 5 June 2019
(This article belongs to the Section Information Theory, Probability and Statistics)
PDF [7470 KB, uploaded 5 June 2019]


In this paper, we present a new effective infrared (IR) and visible (VIS) image fusion method by using a deep neural network. In our method, a Siamese convolutional neural network (CNN) is applied to automatically generate a weight map which represents the saliency of each pixel for a pair of source images. A CNN plays a role in automatic encoding an image into a feature domain for classification. By applying the proposed method, the key problems in image fusion, which are the activity level measurement and fusion rule design, can be figured out in one shot. The fusion is carried out through the multi-scale image decomposition based on wavelet transform, and the reconstruction result is more perceptual to a human visual system. In addition, the visual qualitative effectiveness of the proposed fusion method is evaluated by comparing pedestrian detection results with other methods, by using the YOLOv3 object detector using a public benchmark dataset. The experimental results show that our proposed method showed competitive results in terms of both quantitative assessment and visual quality. View Full-Text
Keywords: image fusion; visible; infrared; convolutional neural network; Siamese network image fusion; visible; infrared; convolutional neural network; Siamese network

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Share & Cite This Article

MDPI and ACS Style

Piao, J.; Chen, Y.; Shin, H. A New Deep Learning Based Multi-Spectral Image Fusion Method. Entropy 2019, 21, 570.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top