Special Issue "Advances in Image Fusion"

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: 6 July 2022 | Viewed by 20037

Special Issue Editors

Prof. Dr. Jiayi Ma
E-Mail Website
Guest Editor
Electronic Information School, Wuhan University, Wuhan 430072, China
Interests: machine learning; computer vision; information fusion; image super resolution; hyperspectral image analysis; infrared imaging; image denoising
Special Issues, Collections and Topics in MDPI journals
Dr. Yu Liu
E-Mail Website
Co-Guest Editor
Department of Biomedical Engineering, Hefei University of Technology, Hefei 230009, China
Interests: image fusion; image super-resolution; visual recognition; biomedical image analysis; machine learning; computer vision
Prof. Dr. Junjun Jiang
E-Mail Website
Co-Guest Editor
School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China
Interests: computer vision; image processing; hyperspectral image analysis; visual recognition; machine learning
Special Issues, Collections and Topics in MDPI journals
Dr. Zheng Wang
E-Mail Website
Co-Guest Editor
Department of Information and Communication Engineering, The University of Tokyo, Tokyo 113-8656, Japan
Interests: person re-identification; image retrieval; crowd counting; image enhancement; multimedia content analysis
Ms. Han Xu
E-Mail Website
Assistant Guest Editor
Electronic Information School, Wuhan University, Wuhan 430070, China
Interests: computer vision; image fusion; deep learning

Special Issue Information

Dear Colleagues,

Many engineering, medical, remote sensing, environmental, national defense, and civilian applications require multiple types of information. Some examples are multimodality images; images with multiple exposure or focus settings; and images including multispectral, hyperspectral, and panchromatic images, ect. A single type of information can merely represent a part of the scene information, while the combination of multiple types of information can provide a comprehensive characterization. However, the deficiency is that the redundant multiple types of information take up much unnecessary storage space. Thus, the challenge of generating aligned and synthesized results by integrating complementary information has gained significant attention, both from storage space and visual perception viewpoints.

The implementation of information fusion is often hindered by different understandings of information, the definitions of meaningful information for subsequent tasks, the ways of information decomposition, the methods used to distinguish complementary information from redundant information, and the design of fusion rules, ect. Further progress on these issues calls for clearer physical explanations of these methods and definitions. Contributions addressing any of these issues are welcome.

This Special Issue aims to be a forum for new and improved information fusion techniques, which are not restricted to image fusion. Considering that there are inevitable offsets in the information collection process and that source images may sometimes be of low quality, it is likely that fusion techniques involving image registration and image enhancement will gain more popularity. In addition, fusion performance evaluation is also a significant factor in designing a good fusion algorithm. Therefore, studies on image quality assessment techniques, such as image-entropy-based methods, are welcome in this Special Issue.

Prof. Dr. Jiayi Ma
Dr. Yu Liu
Prof. Dr. Junjun Jiang
Dr. Zheng Wang
Ms. Han Xu
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • visible and infrared image fusion
  • medical image fusion
  • multi-exposure image fusion
  • multi-focus image fusion
  • remote sensing image fusion
  • image registration
  • image super-resolution
  • image enhancement and image super-resolution
  • image quality assessment
  • information fusion
  • fusion applications

Published Papers (22 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Article
Multi-Focus Image Fusion Based on Multi-Scale Generative Adversarial Network
Entropy 2022, 24(5), 582; https://doi.org/10.3390/e24050582 - 21 Apr 2022
Viewed by 437
Abstract
The methods based on the convolutional neural network have demonstrated its powerful information integration ability in image fusion. However, most of the existing methods based on neural networks are only applied to a part of the fusion process. In this paper, an end-to-end [...] Read more.
The methods based on the convolutional neural network have demonstrated its powerful information integration ability in image fusion. However, most of the existing methods based on neural networks are only applied to a part of the fusion process. In this paper, an end-to-end multi-focus image fusion method based on a multi-scale generative adversarial network (MsGAN) is proposed that makes full use of image features by a combination of multi-scale decomposition with a convolutional neural network. Extensive qualitative and quantitative experiments on the synthetic and Lytro datasets demonstrated the effectiveness and superiority of the proposed MsGAN compared to the state-of-the-art multi-focus image fusion methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
TPFusion: Texture Preserving Fusion of Infrared and Visible Images via Dense Networks
Entropy 2022, 24(2), 294; https://doi.org/10.3390/e24020294 - 19 Feb 2022
Cited by 1 | Viewed by 596
Abstract
In this paper, we design an infrared (IR) and visible (VIS) image fusion via unsupervised dense networks, termed as TPFusion. Activity level measurements and fusion rules are indispensable parts of conventional image fusion methods. However, designing an appropriate fusion process is time-consuming and [...] Read more.
In this paper, we design an infrared (IR) and visible (VIS) image fusion via unsupervised dense networks, termed as TPFusion. Activity level measurements and fusion rules are indispensable parts of conventional image fusion methods. However, designing an appropriate fusion process is time-consuming and complicated. In recent years, deep learning-based methods are proposed to handle this problem. However, for multi-modality image fusion, using the same network cannot extract effective feature maps from source images that are obtained by different image sensors. In TPFusion, we can avoid this issue. At first, we extract the textural information of the source images. Then two densely connected networks are trained to fuse textural information and source image, respectively. By this way, we can preserve more textural details in the fused image. Moreover, loss functions we designed to constrain two densely connected convolutional networks are according to the characteristics of textural information and source images. Through our method, the fused image will obtain more textural information of source images. For proving the validity of our method, we implement comparison and ablation experiments from the qualitative and quantitative assessments. The ablation experiments prove the effectiveness of TPFusion. Being compared to existing advanced IR and VIS image fusion methods, our fusion results possess better fusion results in both objective and subjective aspects. To be specific, in qualitative comparisons, our fusion results have better contrast ratio and abundant textural details. In quantitative comparisons, TPFusion outperforms existing representative fusion methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Multiscale Geometric Analysis Fusion-Based Unsupervised Change Detection in Remote Sensing Images via FLICM Model
Entropy 2022, 24(2), 291; https://doi.org/10.3390/e24020291 - 18 Feb 2022
Cited by 1 | Viewed by 476
Abstract
Remote sensing image change detection is widely used in land use and natural disaster detection. In order to improve the accuracy of change detection, a robust change detection method based on nonsubsampled contourlet transform (NSCT) fusion and fuzzy local information C-means clustering (FLICM) [...] Read more.
Remote sensing image change detection is widely used in land use and natural disaster detection. In order to improve the accuracy of change detection, a robust change detection method based on nonsubsampled contourlet transform (NSCT) fusion and fuzzy local information C-means clustering (FLICM) model is introduced in this paper. Firstly, the log-ratio and mean-ratio operators are used to generate the difference image (DI), respectively; then, the NSCT fusion model is utilized to fuse the two difference images, and one new DI is obtained. The fused DI can not only reflect the real change trend but also suppress the background. The FLICM is performed on the new DI to obtain the final change detection map. Four groups of homogeneous remote sensing images are selected for simulation experiments, and the experimental results demonstrate that the proposed homogeneous change detection method has a superior performance than other state-of-the-art algorithms. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Research on Color Image Encryption Algorithm Based on Bit-Plane and Chen Chaotic System
Entropy 2022, 24(2), 186; https://doi.org/10.3390/e24020186 - 26 Jan 2022
Cited by 1 | Viewed by 770
Abstract
In response to the problems of high complexity and the large amount of operations of existing color image encryption algorithms, a low-complexity, low-operation color image encryption algorithm based on a combination of bit-plane and chaotic systems is proposed that is interrelated with plaintext [...] Read more.
In response to the problems of high complexity and the large amount of operations of existing color image encryption algorithms, a low-complexity, low-operation color image encryption algorithm based on a combination of bit-plane and chaotic systems is proposed that is interrelated with plaintext information. Firstly, three channels of an RGB image are extracted, and the gray value of each pixel channel can be expressed by an eight-bit binary number. The higher- and lower-four bits of the binary gray value of each pixel are exchanged, and the position of each four-bit binary number is scrambled by a logistic chaotic sequence, and all the four-bit binary numbers are converted into hexadecimal numbers to reduce the computational complexity. Next, the position of the transformed image is scrambled by a logistic chaotic sequence. Then, the Chen chaos sequence is used to permute the gray pixel values of the permuted image. Finally, the gray value of the encrypted image is converted into a decimal number to form a single-channel encrypted image, and the three-channel encrypted image is synthesized into an encrypted color image. Through MATLAB simulation experiments, a security analysis of encryption effects in terms of a histogram, correlation, a differential attack, and information entropy is performed. The results show that the algorithm has a better encryption effect and is resistant to differential attacks. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Synthetic Aperture Radar Image Despeckling Based on Multi-Weighted Sparse Coding
Entropy 2022, 24(1), 96; https://doi.org/10.3390/e24010096 - 07 Jan 2022
Viewed by 309
Abstract
Synthetic aperture radar (SAR) images are inherently degraded by speckle noise caused by coherent imaging, which may affect the performance of the subsequent image analysis task. To resolve this problem, this article proposes an integrated SAR image despeckling model based on dictionary learning [...] Read more.
Synthetic aperture radar (SAR) images are inherently degraded by speckle noise caused by coherent imaging, which may affect the performance of the subsequent image analysis task. To resolve this problem, this article proposes an integrated SAR image despeckling model based on dictionary learning and multi-weighted sparse coding. First, the dictionary is trained by groups composed of similar image patches, which have the same structural features. An effective orthogonal dictionary with high sparse representation ability is realized by introducing a properly tight frame. Furthermore, the data-fidelity term and regularization terms are constrained by weighting factors. The weighted sparse representation model not only fully utilizes the interblock relevance but also reflects the importance of various structural groups in despeckling processing. The proposed model is implemented with fast and effective solving steps that simultaneously perform orthogonal dictionary learning, weight parameter updating, sparse coding, and image reconstruction. The solving steps are designed using the alternative minimization method. Finally, the speckles are further suppressed by iterative regularization methods. In a comparison study with existing methods, our method demonstrated state-of-the-art performance in suppressing speckle noise and protecting the image texture details. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
S2A: Scale-Attention-Aware Networks for Video Super-Resolution
Entropy 2021, 23(11), 1398; https://doi.org/10.3390/e23111398 - 25 Oct 2021
Cited by 1 | Viewed by 528
Abstract
Convolutional Neural Networks (CNNs) have been widely used in video super-resolution (VSR). Most existing VSR methods focus on how to utilize the information of multiple frames, while neglecting the feature correlations of the intermediate features, thus limiting the feature expression of the models. [...] Read more.
Convolutional Neural Networks (CNNs) have been widely used in video super-resolution (VSR). Most existing VSR methods focus on how to utilize the information of multiple frames, while neglecting the feature correlations of the intermediate features, thus limiting the feature expression of the models. To address this problem, we propose a novel SAA network, that is, Scale-and-Attention-Aware Networks, to apply different attention to different temporal-length streams, while further exploring both spatial and channel attention on separate streams with a newly proposed Criss-Cross Channel Attention Module (C3AM). Experiments on public VSR datasets demonstrate the superiority of our method over other state-of-the-art methods in terms of both quantitative and qualitative metrics. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Multi-Focus Image Fusion Method Based on Multi-Scale Decomposition of Information Complementary
Entropy 2021, 23(10), 1362; https://doi.org/10.3390/e23101362 - 19 Oct 2021
Viewed by 612
Abstract
Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially [...] Read more.
Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially when the source images captured by cameras produce anisotropic blur and unregistration. This paper proposes a new multi-focus image fusion method based on the multi-scale decomposition of complementary information. Firstly, this method uses two groups of large-scale and small-scale decomposition schemes that are structurally complementary, to perform two-scale double-layer singular value decomposition of the image separately and obtain low-frequency and high-frequency components. Then, the low-frequency components are fused by a rule that integrates image local energy with edge energy. The high-frequency components are fused by the parameter-adaptive pulse-coupled neural network model (PA-PCNN), and according to the feature information contained in each decomposition layer of the high-frequency components, different detailed features are selected as the external stimulus input of the PA-PCNN. Finally, according to the two-scale decomposition of the source image that is structure complementary, and the fusion of high and low frequency components, two initial decision maps with complementary information are obtained. By refining the initial decision graph, the final fusion decision map is obtained to complete the image fusion. In addition, the proposed method is compared with 10 state-of-the-art approaches to verify its effectiveness. The experimental results show that the proposed method can more accurately distinguish the focused and non-focused areas in the case of image pre-registration and unregistration, and the subjective and objective evaluation indicators are slightly better than those of the existing methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
A New Variational Bayesian-Based Kalman Filter with Unknown Time-Varying Measurement Loss Probability and Non-Stationary Heavy-Tailed Measurement Noise
Entropy 2021, 23(10), 1351; https://doi.org/10.3390/e23101351 - 16 Oct 2021
Viewed by 566
Abstract
In this paper, a new variational Bayesian-based Kalman filter (KF) is presented to solve the filtering problem for a linear system with unknown time-varying measurement loss probability (UTVMLP) and non-stationary heavy-tailed measurement noise (NSHTMN). Firstly, the NSHTMN was modelled as a Gaussian-Student’s t [...] Read more.
In this paper, a new variational Bayesian-based Kalman filter (KF) is presented to solve the filtering problem for a linear system with unknown time-varying measurement loss probability (UTVMLP) and non-stationary heavy-tailed measurement noise (NSHTMN). Firstly, the NSHTMN was modelled as a Gaussian-Student’s t-mixture distribution via employing a Bernoulli random variable (BM). Secondly, by utilizing another Bernoulli random variable (BL), the form of the likelihood function consisting of two mixture distributions was converted from a weight sum to an exponential product and a new hierarchical Gaussian state-space model was therefore established. Finally, the system state vector, BM, BL, the intermediate random variables, the mixing probability, and the UTVMLP were jointly inferred by employing the variational Bayesian technique. Simulation results revealed that in the scenario of NSHTMN, the proposed filter had a better performance than current algorithms and further improved the estimation accuracy of UTVMLP. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
A Multi-Modal Fusion Method Based on Higher-Order Orthogonal Iteration Decomposition
Entropy 2021, 23(10), 1349; https://doi.org/10.3390/e23101349 - 15 Oct 2021
Cited by 1 | Viewed by 443
Abstract
Multi-modal fusion can achieve better predictions through the amalgamation of information from different modalities. To improve the performance of accuracy, a method based on Higher-order Orthogonal Iteration Decomposition and Projection (HOIDP) is proposed, in the fusion process, higher-order orthogonal iteration decomposition algorithm and [...] Read more.
Multi-modal fusion can achieve better predictions through the amalgamation of information from different modalities. To improve the performance of accuracy, a method based on Higher-order Orthogonal Iteration Decomposition and Projection (HOIDP) is proposed, in the fusion process, higher-order orthogonal iteration decomposition algorithm and factor matrix projection are used to remove redundant information duplicated inter-modal and produce fewer parameters with minimal information loss. The performance of the proposed method is verified by three different multi-modal datasets. The numerical results validate the accuracy of the performance of the proposed method having 0.4% to 4% improvement in sentiment analysis, 0.3% to 8% improvement in personality trait recognition, and 0.2% to 25% improvement in emotion recognition at three different multi-modal datasets compared with other 5 methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Learning Two-View Correspondences and Geometry via Local Neighborhood Correlation
Entropy 2021, 23(8), 1024; https://doi.org/10.3390/e23081024 - 09 Aug 2021
Viewed by 675
Abstract
Seeking quality feature correspondences (also known as matches) is a foundational step in computer vision. In our work, a novel and effective network with a stable local constraint, named the Local Neighborhood Correlation Network (LNCNet), is proposed to capture abundant contextual information of [...] Read more.
Seeking quality feature correspondences (also known as matches) is a foundational step in computer vision. In our work, a novel and effective network with a stable local constraint, named the Local Neighborhood Correlation Network (LNCNet), is proposed to capture abundant contextual information of each correspondence in the local region, followed by calculating the essential matrix and camera pose estimation. Firstly, the k-Nearest Neighbor (KNN) algorithm is used to divide the local neighborhood roughly. Then, we calculate the local neighborhood correlation matrix (LNC) between the selected correspondence and other correspondences in the local region, which is used to filter outliers to obtain more accurate local neighborhood information. We cluster the filtered information into feature vectors containing richer neighborhood contextual information so that they can be used to more accurately determine the probability of correspondences as inliers. Extensive experiments have demonstrated that our proposed LNCNet performs better than some state-of-the-art networks to accomplish outlier rejection and camera pose estimation tasks in complex outdoor and indoor scenes. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
News Video Summarization Combining SURF and Color Histogram Features
Entropy 2021, 23(8), 982; https://doi.org/10.3390/e23080982 - 30 Jul 2021
Cited by 1 | Viewed by 683
Abstract
Because the data volume of news videos is increasing exponentially, a way to quickly browse a sketch of the video is important in various applications, such as news media, archives and publicity. This paper proposes a news video summarization method based on SURF [...] Read more.
Because the data volume of news videos is increasing exponentially, a way to quickly browse a sketch of the video is important in various applications, such as news media, archives and publicity. This paper proposes a news video summarization method based on SURF features and an improved clustering algorithm, to overcome the defects in existing algorithms that fail to account for changes in shot complexity. Firstly, we extracted SURF features from the video sequences and matched the features between adjacent frames, and then detected the abrupt and gradual boundaries of the shot by calculating similarity scores between adjacent frames with the help of double thresholds. Secondly, we used an improved clustering algorithm to cluster the color histogram of the video frames within the shot, which merged the smaller clusters and then selected the frame closest to the cluster center as the key frame. The experimental results on both the public and self-built datasets show the superiority of our method over the alternatives in terms of accuracy and speed. Additionally, the extracted key frames demonstrate low redundancy and can credibly represent a sketch of news videos. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Pairwise Elastic Net Representation-Based Classification for Hyperspectral Image Classification
Entropy 2021, 23(8), 956; https://doi.org/10.3390/e23080956 - 26 Jul 2021
Cited by 2 | Viewed by 627
Abstract
The representation-based algorithm has raised a great interest in hyperspectral image (HSI) classification. l1-minimization-based sparse representation (SR) attempts to select a few atoms and cannot fully reflect within-class information, while l2-minimization-based collaborative representation (CR) tries to use all of [...] Read more.
The representation-based algorithm has raised a great interest in hyperspectral image (HSI) classification. l1-minimization-based sparse representation (SR) attempts to select a few atoms and cannot fully reflect within-class information, while l2-minimization-based collaborative representation (CR) tries to use all of the atoms leading to mixed-class information. Considering the above problems, we propose the pairwise elastic net representation-based classification (PENRC) method. PENRC combines the l1-norm and l2-norm penalties and introduces a new penalty term, including a similar matrix between dictionary atoms. This similar matrix enables the automatic grouping selection of highly correlated data to estimate more robust weight coefficients for better classification performance. To reduce computation cost and further improve classification accuracy, we use part of the atoms as a local adaptive dictionary rather than the entire training atoms. Furthermore, we consider the neighbor information of each pixel and propose a joint pairwise elastic net representation-based classification (J-PENRC) method. Experimental results on chosen hyperspectral data sets confirm that our proposed algorithms outperform the other state-of-the-art algorithms. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Robust Vehicle Speed Measurement Based on Feature Information Fusion for Vehicle Multi-Characteristic Detection
Entropy 2021, 23(7), 910; https://doi.org/10.3390/e23070910 - 17 Jul 2021
Cited by 3 | Viewed by 1490
Abstract
A robust vehicle speed measurement system based on feature information fusion for vehicle multi-characteristic detection is proposed in this paper. A vehicle multi-characteristic dataset is constructed. With this dataset, seven CNN-based modern object detection algorithms are trained for vehicle multi-characteristic detection. The FPN-based [...] Read more.
A robust vehicle speed measurement system based on feature information fusion for vehicle multi-characteristic detection is proposed in this paper. A vehicle multi-characteristic dataset is constructed. With this dataset, seven CNN-based modern object detection algorithms are trained for vehicle multi-characteristic detection. The FPN-based YOLOv4 is selected as the best vehicle multi-characteristic detection algorithm, which applies feature information fusion of different scales with both rich high-level semantic information and detailed low-level location information. The YOLOv4 algorithm is improved by combing with the attention mechanism, in which the residual module in YOLOv4 is replaced by the ECA channel attention module with cross channel interaction. An improved ECA-YOLOv4 object detection algorithm based on both feature information fusion and cross channel interaction is proposed, which improves the performance of YOLOv4 for vehicle multi-characteristic detection and reduces the model parameter size and FLOPs as well. A multi-characteristic fused speed measurement system based on license plate, logo, and light is designed accordingly. The system performance is verified by experiments. The experimental results show that the speed measurement error rate of the proposed system meets the requirement of the China national standard GB/T 21555-2007 in which the speed measurement error rate should be less than 6%. The proposed system can efficiently enhance the vehicle speed measurement accuracy and effectively improve the vehicle speed measurement robustness. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
An Improved Stereo Matching Algorithm for Vehicle Speed Measurement System Based on Spatial and Temporal Image Fusion
Entropy 2021, 23(7), 866; https://doi.org/10.3390/e23070866 - 07 Jul 2021
Cited by 1 | Viewed by 753
Abstract
This paper proposes an improved stereo matching algorithm for vehicle speed measurement system based on spatial and temporal image fusion (STIF). Firstly, the matching point pairs in the license plate area with obviously abnormal distance to the camera are roughly removed according to [...] Read more.
This paper proposes an improved stereo matching algorithm for vehicle speed measurement system based on spatial and temporal image fusion (STIF). Firstly, the matching point pairs in the license plate area with obviously abnormal distance to the camera are roughly removed according to the characteristic of license plate specification. Secondly, more mismatching point pairs are finely removed according to local neighborhood consistency constraint (LNCC). Thirdly, the optimum speed measurement point pairs are selected for successive stereo frame pairs by STIF of binocular stereo video, so that the 3D points corresponding to the matching point pairs for speed measurement in the successive stereo frame pairs are in the same position on the real vehicle, which can significantly improve the vehicle speed measurement accuracy. LNCC and STIF can be used not only for license plate, but also for vehicle logo, light, mirror etc. Experimental results demonstrate that the vehicle speed measurement system with the proposed LNCC+STIF stereo matching algorithm can significantly outperform the state-of-the-art system in accuracy. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
A Foreground-Aware Framework for Local Face Attribute Transfer
Entropy 2021, 23(5), 615; https://doi.org/10.3390/e23050615 - 16 May 2021
Viewed by 795
Abstract
In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there [...] Read more.
In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there are many algorithms automatically and globally transferring the style from one image to another, they fail to respect the semantics of the scene and are unable to allow users to merely transfer the attributes of one or two face organs in the foreground region leaving the background region unchanged. To overcome this problem, we developed a novel framework for semantically meaningful local face attribute transfer, which can flexibly transfer the local attribute of a face organ from the reference image to a semantically equivalent organ in the input image, while preserving the background. Our method involves warping the reference photo to match the shape, pose, location, and expression of the input image. The fusion of the warped reference image and input image is then taken as the initialized image for a neural style transfer algorithm. Our method achieves better performance in terms of inception score (3.81) and Fréchet inception distance (80.31), which is about 10% higher than those of competitors, indicating that our framework is capable of producing high-quality and photorealistic attribute transfer results. Both theoretical findings and experimental results are provided to demonstrate the efficacy of the proposed framework, reveal its superiority over other state-of-the-art alternatives. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain
Entropy 2021, 23(5), 591; https://doi.org/10.3390/e23050591 - 11 May 2021
Cited by 5 | Viewed by 784
Abstract
Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering [...] Read more.
Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering (GIF) in non-subsampled contourlet transform (NSCT) domain. Firstly, the source images are decomposed by NSCT, several low- and high-frequency sub-bands are generated. Secondly, the PCNN-based fusion rule is used to process the low-frequency components, and the GIF-WSEML fusion model is used to process the high-frequency components. Finally, the fused image is obtained by integrating the fused low- and high-frequency sub-bands. The experimental results demonstrate that the proposed method can achieve better performance in terms of multimodal medical image fusion. The proposed algorithm also has obvious advantages in objective evaluation indexes VIFF, QW, API, SD, EN and time consumption. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Unsupervised Exemplar-Domain Aware Image-to-Image Translation
Entropy 2021, 23(5), 565; https://doi.org/10.3390/e23050565 - 02 May 2021
Cited by 2 | Viewed by 845
Abstract
Image-to-image translation is used to convert an image of a certain style to another of the target style with the original content preserved. A desired translator should be capable of generating diverse results in a controllable many-to-many fashion. To this end, we design [...] Read more.
Image-to-image translation is used to convert an image of a certain style to another of the target style with the original content preserved. A desired translator should be capable of generating diverse results in a controllable many-to-many fashion. To this end, we design a novel deep translator, namely exemplar-domain aware image-to-image translator (EDIT for short). From a logical perspective, the translator needs to perform two main functions, i.e., feature extraction and style transfer. With consideration of logical network partition, the generator of our EDIT comprises of a part of blocks configured by shared parameters, and the rest by varied parameters exported by an exemplar-domain aware parameter network, for explicitly imitating the functionalities of extraction and mapping. The principle behind this is that, for images from multiple domains, the content features can be obtained by an extractor, while (re-)stylization is achieved by mapping the extracted features specifically to different purposes (domains and exemplars). In addition, a discriminator is equipped during the training phase to guarantee the output satisfying the distribution of the target domain. Our EDIT can flexibly and effectively work on multiple domains and arbitrary exemplars in a unified neat model. We conduct experiments to show the efficacy of our design, and reveal its advances over other state-of-the-art methods both quantitatively and qualitatively. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
A Noisy SAR Image Fusion Method Based on NLM and GAN
Entropy 2021, 23(4), 410; https://doi.org/10.3390/e23040410 - 30 Mar 2021
Viewed by 760
Abstract
The unavoidable noise often present in synthetic aperture radar (SAR) images, such as speckle noise, negatively impacts the subsequent processing of SAR images. Further, it is not easy to find an appropriate application for SAR images, given that the human visual system is [...] Read more.
The unavoidable noise often present in synthetic aperture radar (SAR) images, such as speckle noise, negatively impacts the subsequent processing of SAR images. Further, it is not easy to find an appropriate application for SAR images, given that the human visual system is sensitive to color and SAR images are gray. As a result, a noisy SAR image fusion method based on nonlocal matching and generative adversarial networks is presented in this paper. A nonlocal matching method is applied to processing source images into similar block groups in the pre-processing step. Then, adversarial networks are employed to generate a final noise-free fused SAR image block, where the generator aims to generate a noise-free SAR image block with color information, and the discriminator tries to increase the spatial resolution of the generated image block. This step ensures that the fused image block contains high resolution and color information at the same time. Finally, a fused image can be obtained by aggregating all the image blocks. By extensive comparative experiments on the SEN1–2 datasets and source images, it can be found that the proposed method not only has better fusion results but is also robust to image noise, indicating the superiority of the proposed noisy SAR image fusion method over the state-of-the-art methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation
Entropy 2021, 23(3), 376; https://doi.org/10.3390/e23030376 - 21 Mar 2021
Cited by 6 | Viewed by 1496
Abstract
This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided [...] Read more.
This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided into foregrounds and backgrounds by semantic masks. The generator with a dual-encoder-single-decoder framework is used to extract the feature of foregrounds and backgrounds by different encoder paths. Moreover, the discriminator’s input image is designed based on semantic segmentation, which is obtained by combining the foregrounds of the infrared images with the backgrounds of the visible images. Consequently, the prominence of thermal targets in the infrared images and texture details in the visible images can be preserved in the fused images simultaneously. Qualitative and quantitative experiments on publicly available datasets demonstrate that the proposed approach can significantly outperform the state-of-the-art methods. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
An Improvised Machine Learning Model Based on Mutual Information Feature Selection Approach for Microbes Classification
Entropy 2021, 23(2), 257; https://doi.org/10.3390/e23020257 - 23 Feb 2021
Cited by 2 | Viewed by 1794
Abstract
The accurate classification of microbes is critical in today’s context for monitoring the ecological balance of a habitat. Hence, in this research work, a novel method to automate the process of identifying microorganisms has been implemented. To extract the bodies of microorganisms accurately, [...] Read more.
The accurate classification of microbes is critical in today’s context for monitoring the ecological balance of a habitat. Hence, in this research work, a novel method to automate the process of identifying microorganisms has been implemented. To extract the bodies of microorganisms accurately, a generalized segmentation mechanism which consists of a combination of convolution filter (Kirsch) and a variance-based pixel clustering algorithm (Otsu) is proposed. With exhaustive corroboration, a set of twenty-five features were identified to map the characteristics and morphology for all kinds of microbes. Multiple techniques for feature selection were tested and it was found that mutual information (MI)-based models gave the best performance. Exhaustive hyperparameter tuning of multilayer layer perceptron (MLP), k-nearest neighbors (KNN), quadratic discriminant analysis (QDA), logistic regression (LR), and support vector machine (SVM) was done. It was found that SVM radial required further improvisation to attain a maximum possible level of accuracy. Comparative analysis between SVM and improvised SVM (ISVM) through a 10-fold cross validation method ultimately showed that ISVM resulted in a 2% higher performance in terms of accuracy (98.2%), precision (98.2%), recall (98.1%), and F1 score (98.1%). Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Exploiting Superpixels for Multi-Focus Image Fusion
Entropy 2021, 23(2), 247; https://doi.org/10.3390/e23020247 - 21 Feb 2021
Cited by 3 | Viewed by 1157
Abstract
Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. [...] Read more.
Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. This makes it useful for numerous applications in image enhancement, remote sensing, object recognition, medical imaging, etc. This paper presents a novel multi-focus image fusion algorithm that proposes to group the local connected pixels with similar colors and patterns, usually referred to as superpixels, and use them to separate the focused and de-focused regions of an image. We note that these superpixels are more expressive than individual pixels, and they carry more distinctive statistical properties when compared with other superpixels. The statistical properties of superpixels are analyzed to categorize the pixels as focused or de-focused and to estimate a focus map. A spatial consistency constraint is ensured on the initial focus map to obtain a refined map, which is used in the fusion rule to obtain a single all-in-focus image. Qualitative and quantitative evaluations are performed to assess the performance of the proposed method on a benchmark multi-focus image fusion dataset. The results show that our method produces better quality fused images than existing image fusion techniques. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Article
Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images
Entropy 2021, 23(2), 239; https://doi.org/10.3390/e23020239 - 19 Feb 2021
Cited by 1 | Viewed by 979
Abstract
Obtaining key and rich visual information under sophisticated road conditions is one of the key requirements for advanced driving assistance. In this paper, a newfangled end-to-end model is proposed for advanced driving assistance based on the fusion of infrared and visible images, termed [...] Read more.
Obtaining key and rich visual information under sophisticated road conditions is one of the key requirements for advanced driving assistance. In this paper, a newfangled end-to-end model is proposed for advanced driving assistance based on the fusion of infrared and visible images, termed as FusionADA. In our model, we are committed to extracting and fusing the optimal texture details and salient thermal targets from the source images. To achieve this goal, our model constitutes an adversarial framework between the generator and the discriminator. Specifically, the generator aims to generate a fused image with basic intensity information together with the optimal texture details from source images, while the discriminator aims to force the fused image to restore the salient thermal targets from the source infrared image. In addition, our FusionADA is a fully end-to-end model, solving the issues of manually designing complicated activity level measurements and fusion rules existing in traditional methods. Qualitative and quantitative experiments on publicly available datasets RoadScene and TNO demonstrate the superiority of our FusionADA over the state-of-the-art approaches. Full article
(This article belongs to the Special Issue Advances in Image Fusion)
Show Figures

Figure 1

Back to TopTop