Next Article in Journal
Simulation of Fresnel Zone Plate Imaging Performance with Number of Zones
Previous Article in Journal
Flexible Sensor Array Based on Transient Earth Voltage for Online Partial Discharge Monitoring of Cable Termination
Open AccessArticle

Structural Similarity Loss for Learning to Fuse Multi-Focus Images

1
School of Physics and Optoelectronic Engineering, Xidian University, Xi’an 710071, China
2
School of Science, Edith Cowan University, Joondalup, WA 6027, Australia
3
Computer Science and Software Engineering, The University of Western Australia, Crawley, WA 6009, Australia
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(22), 6647; https://doi.org/10.3390/s20226647
Received: 3 October 2020 / Revised: 29 October 2020 / Accepted: 17 November 2020 / Published: 20 November 2020
(This article belongs to the Section Sensing and Imaging)
Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as ‘focused’ or ‘defocused’, and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks. View Full-Text
Keywords: multi-focus image fusion; convolution neural network; unsupervised learning; structural similarity multi-focus image fusion; convolution neural network; unsupervised learning; structural similarity
Show Figures

Figure 1

MDPI and ACS Style

Yan, X.; Gilani, S.Z.; Qin, H.; Mian, A. Structural Similarity Loss for Learning to Fuse Multi-Focus Images. Sensors 2020, 20, 6647. https://doi.org/10.3390/s20226647

AMA Style

Yan X, Gilani SZ, Qin H, Mian A. Structural Similarity Loss for Learning to Fuse Multi-Focus Images. Sensors. 2020; 20(22):6647. https://doi.org/10.3390/s20226647

Chicago/Turabian Style

Yan, Xiang; Gilani, Syed Z.; Qin, Hanlin; Mian, Ajmal. 2020. "Structural Similarity Loss for Learning to Fuse Multi-Focus Images" Sensors 20, no. 22: 6647. https://doi.org/10.3390/s20226647

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop