Gaussian of Differences: A Simple and Efficient General Image Fusion Method

The separate analysis of images obtained from a single source using different camera settings or spectral bands, whether from one or more than one sensor, is quite difficult. To solve this problem, a single image containing all of the distinctive pieces of information in each source image is generally created by combining the images, a process called image fusion. In this paper, a simple and efficient, pixel-based image fusion method is proposed that relies on weighting the edge information associated with each pixel of all of the source images proportional to the distance from their neighbors by employing a Gaussian filter. The proposed method, Gaussian of differences (GD), was evaluated using multi-modal medical images, multi-sensor visible and infrared images, multi-focus images, and multi-exposure images, and was compared to existing state-of-the-art fusion methods by utilizing objective fusion quality metrics. The parameters of the GD method are further enhanced by employing the pattern search (PS) algorithm, resulting in an adaptive optimization strategy. Extensive experiments illustrated that the proposed GD fusion method ranked better on average than others in terms of objective quality metrics and CPU time consumption.


Introduction
The objective of image fusion is to merge the complementary information derived from multiple source images into a unified image [1][2][3][4].In multi-modal medical image fusion, two or more images from different imaging modalities are combined [5].Magnetic resonance (MR) and computed tomography (CT) are two different medical imaging modalities that have complementary strengths and weaknesses.CT images have high spatial resolution, which makes bones more visible, while MR images have high contrast resolution, which reveals soft tissues such as organs [6].Visible and infrared image fusion is a computational technique that includes combined information from infrared and visible spectrum images to improve the visibility of objects and enhance the contrast of images, especially for enhanced night vision, remote sensing and pan-sharpening [7][8][9][10][11][12].Multi-exposure image fusion involves the integration of multiple images, each captured at varying exposure levels, to generate a high-dynamic-range (HDR) image.HDR images retain details in both the dark and bright regions, which enhances image quality, increases visual fidelity, and improves image analysis in computer vision tasks [13,14].Multi-focus image fusion is employed to merge multiple images exhibiting distinct focus levels into a singular composite image [15][16][17][18][19].This results in improved overall sharpness, enhanced depth of field, and enhanced visual perception [20].These benefits enable more accurate analysis and interpretation of the fused image in computer vision applications.

Related Work
Image fusion methods in the literature can be basically divided into two categories: pixel domain and transformation domain [21].Pixel-domain (or spatial-domain) tech-Entropy 2023, 25, 1215 2 of 31 niques combine the source images directly using their gray-level or color pixel values.The best-known example of this technique is the arithmetic averaging of source images.Arithmetic averaging can be used to combine both multi-sensor and multifocal images, but the biggest disadvantage of this method is that it reduces image contrast [22].The basic idea of multi-scale, transform-based image fusion methods is applying a multi-resolution decomposition to each source image, combining the decomposition results with various rules to create a unified representation, and finally, applying an inverse multi-resolution transform [23].Well-known examples of these approaches include principal component analysis (PCA), discrete wavelet transform (DWT), Laplacian pyramid (LP), and other pyramid-based transformations [24].In recent years, several image fusion algorithms based on machine learning and deep learning approaches have been proposed [3,[25][26][27][28].These methods are robust and demonstrate superior performance.However, the training phase requires powerful, high-performance computing systems and plenty of input training data.Moreover, the trained models can be time-consuming for real-time applications [29].
Pixel level, feature level, and decision level are the three levels at which image fusion can take place.Pixel-level fusion directly integrates the original data from the source images to produce a fused image that is more informative for both computer processing and human visual perception.Compared to other fusion approaches, this approach strives to improve the visual quality and computing efficiency of the fused image.Li et al. proposed a pixelbased method by calculating the pixel visibility for each pixel in the source images [30].Yang and Li proposed a multi-focus image fusion method based on spatial-frequency and morphologic operators [31].Typically, in pixel-level image fusion, the weights are determined based on the activity level of various pixels [32].In these studies, neural networks [33] and support vector machines [34] are employed to select pixels with the most significant activity, using wavelet coefficients as the input features.Ludusan and Lavialle proposed a variational pixel-based method for image fusion based on error estimation theory and partial differential equations to mitigate the noise of images [35].In [36], a technique for multi-exposure image fusion is introduced which involves two primary stages: image features, including local contrast, brightness, and color dissimilarity, are computed to generate weight maps that are further improved using recursive filtering.Subsequently, the fused image is formed by combining the source images using a weighted sum based on these refined weight maps.Besides the many pixel-level methods available, region-based spatial methods that use blocks [37] or adaptive regions [38,39] have also been proposed to outperform existing methods.
Within the framework of anisotropic diffusion filter (ADF)-based image fusion algorithms, weight map layers are formed via image smoothing, which employs an edge protection method.These weight map layers undergo subsequent processing prior to the application of the fusion rule, culminating in the attainment of the final output [40].Kumar has introduced the cross-binary filter (CBF) method, which takes into account both the gray-level similarity and geometric closeness of neighboring pixels without antialiasing.The source images are combined according to the weighted average, using the weights calculated from the detailed images extracted from the source images by the CBF method [41].The fourth-order partial differential equations (FDPE) method first applies differential equations to each source image to obtain approximate images.Then, PCA is used to obtain optimum weights for the detailed images, which are then combined to obtain the final, detailed image.The ultimate approximation of the image is derived by performing an averaging operation on the set of approximate images.Subsequently, the fused image is computed by merging the final approximation with the detailed images [42].The context enhancement (GFCE)-based method preserves the details in the visible input image and the background scene.Thus, it can successfully transfer important IR information to the composite image [43].The gradient transfer fusion (GTF) method, which is based on gradient transmission and total variation (TV) minimization, tries to maintain appearance information and thermal radiation simultaneously [44].The hybrid multi-scale decomposition method (HMSD) decomposes the source image into very distant Entropy 2023, 25, 1215 3 of 31 texture details and edge features using a combination of bilateral filters and the versatile Gaussian method.This offset allows us to better capture important very sensitive IR spectral features and separate fine texture details from large edges [45].The infrared feature extraction and visual information preservation (IFEVIP) method provides a simple, fast, but effective fusion of infrared and visual images.Firstly, the reconstruction of the infrared background is accomplished by leveraging quadtree decomposition and Bézier interpolation.Subsequently, the extraction of bright infrared features is performed by subtracting the reconstructed background from the infrared image, followed by a refinement process that reduces redundant background information [46].The multi-resolution singular value decomposition (MSVD) method is an image fusion technique based on a process that bears a resemblance to wavelet transform and involves filtering the signal independently using low-pass and high-pass finite impulse response (FIR) filters, followed by the decimation of the output of each filter by a factor of two to achieve the first level of decomposition [47].The VSMWLS approach, designed to enhance the transfer of significant visual details while minimizing the inclusion of irrelevant infrared (IR) details or noise in the merged image, represents a multi-scale fusion technique that incorporates visual salience maps (VSM) and weighted least square (WLS) optimization [48].Liu et al. proposed an approach based on deep convolutional neural networks (CNN) for both infrared-visible image fusion [49] and multi-focus image fusion [50].They successfully addressed the crucial issues of activity level measurement and weight assignment in image fusion by using a Siamese convolutional network to construct a weight map by integrating pixel activity information from two source images [49].On the other hand, because focus estimation and image fusion are two distinct problems, traditional image fusion techniques sometimes struggle to perform satisfactorily.Liu et al. suggest a deep learning method that avoids the requirement for separate focus estimation by learning a direct mapping between source images and a focus map [50].

Contributions of This Study and Advantages of the Proposed Method
To overcome the limitations of the existing image fusion methods, a simple and efficient general image fusion technique named Gaussian of differences (GD) is proposed.The unique aspects of the proposed GD image fusion method can be listed as follows:

•
The proposed algorithm does not use any transformations and works directly in the pixel domain.Also, it is based on basic image convolution and linear weighting, which makes it simple and efficient.It can be implemented on real-time systems and is suitable for parallel processing.

•
The method enhances the high-frequency components of each input image using simple first-order derivative edge detection.It then uses a Gaussian filter to weight the contributions of neighboring pixels to the center pixel, with the weight decreasing with distance.

•
The proposed GD method has only two control parameters: the size of the filter and the standard deviation of the distribution.In addition to making use of predefined parameters, an optimal solution using the pattern search (PS) algorithm is also proposed to investigate the adaptability capability of the GD method.

•
The method is a general-purpose image fusion algorithm that can be used in a variety of applications, including multi-modal medical image fusion, infrared and visible image fusion for enhanced night vision or remote sensing, multi-focus image fusion for extending the depth of field, and multi-exposure image fusion for high dynamic range imaging.

•
It can combine single-band (gray-level), color (RGB), multi-spectral, and hyperspectral images due to its generalized structure.
The rest of this paper is organized as follows: the proposed GD fusion method is briefly introduced, illustrated, and demonstrated in Section 2. Section 3 outlines extensive experiments with 48 pairs of test images (in total) belonging to four different image fusion applications.Finally, Section 4 concludes the paper.

Proposed Method
Speed and performance are crucial features of imaging systems.Therefore, one of the primary factors considered in designing the proposed image fusion method was keeping the computational complexity low.Another significant concern was the generation of a single composite image that incorporates meaningful information from images captured at multiple or diverse wavelengths [51].The resulting combined image should be suitable for both human interaction and computer vision applications [52].
Many of the existing fusion methods in the literature employ multi-resolution transforms such as DWT, LP, and discrete cosine transform (DCT) to mitigate the impact of image misalignments [53].However, these transformations increase the computational complexity of the methods.Edge information, which typically contains high-frequency components, plays a crucial role in determining the importance of pixels in an image.
For the method proposed in this paper, at first, the gradients of each source image based on the first-degree derivative information are computed.These gradients are then evaluated along with the neighboring pixels.Linearly, the contribution of each pixel from different input images to the resulting pixel in the final fused image is determined.The block diagram of the proposed GD image fusion method is presented in Figure 1.
Speed and performance are crucial features of imaging systems.Therefore, one o primary factors considered in designing the proposed image fusion method was kee the computational complexity low.Another significant concern was the generation single composite image that incorporates meaningful information from images capt at multiple or diverse wavelengths [51].The resulting combined image should be sui for both human interaction and computer vision applications [52].
Many of the existing fusion methods in the literature employ multi-resolution tr forms such as DWT, LP, and discrete cosine transform (DCT) to mitigate the impa image misalignments [53].However, these transformations increase the computati complexity of the methods.Edge information, which typically contains high-frequ components, plays a crucial role in determining the importance of pixels in an image For the method proposed in this paper, at first, the gradients of each source im based on the first-degree derivative information are computed.These gradients are evaluated along with the neighboring pixels.Linearly, the contribution of each pixel different input images to the resulting pixel in the final fused image is determined.block diagram of the proposed GD image fusion method is presented in Figure 1.The steps of the proposed GD fusion method can be summarized as follows: 1. Edge information is generally related with the information content of an image.
first-order derivation (difference of adjacent pixels) of an image simply empha the edges.The column and row differences of each input image are calculated: where i and j are row and column indexes, CD and RD indicate the column and row The steps of the proposed GD fusion method can be summarized as follows: 1.
Edge information is generally related with the information content of an image.The first-order derivation (difference of adjacent pixels) of an image simply emphasizes the edges.The column and row differences of each input image are calculated: Entropy 2023, 25, 1215 5 of 31 where i and j are row and column indexes, CD and RD indicate the column and row differences, respectively, and k is the input image index.In Figure 2, a face image in the visible spectrum is given as I 1 and an infrared image of the same scene is given as I 2 .The column and row differences of the input images are also visualized.

2.
Column and row differences emphasize the edges along vertical and horizontal axes, respectively.To combine them into a single representation (D), the Euclidian distance is used, and features related with each pixel based on the edge content are calculated (visualized in Figure 3): Linear weighting is a well-known approach used to determine the information transfer of each input image to the output fused image.To determine the contributions of neighbors of pixels in each image at different input images to the information content of the respective pixel, the differences are filtered (i.e., weighted) using a 2D Gaussian filter and the Gaussian of the Differences is obtained (GD), which is visualized in Figure 4.This representation will be used to calculate the weighting factor of each pixel: where s is the window size, w is a 2D Gaussian filter with a standard deviation of σ: Weighting factors (fw) are determined for the pixels in each input image using GD proportional to their values, as visualized in Figure 5. Therefore, the sum of the weighting coefficients of a specific pixel is always equal to one, regardless of how many input images exist: The fused image (F), as demonstrated in Figure 6, is created with the linear weighting method using weighting factors.Assume that there are two input images in an application, and for a specific pixel, let the fws be 0.4 and 0.6, respectively.The fusion result of that specific pixel is summation of 40% of the first input image's pixel value I 1 (i, j) and 60% of the second input image I 2 (i, j).
In the prosed GD fusion method, before calculating the contribution of pixels to the fused image, the placement of the Gaussian filter (7 × 7 for s = 3) is used to contribute to the edge information of each pixel.This is given in Figure 7.The pixel of interest in the center is weighted with the highest coefficient w(0,0) in the Gaussian kernel, and the neighbors are weighted with smaller coefficients as they move away from the center due to the nature of the Gaussian kernel.The fusion results are promising, as shown in the visual steps of the proposed GD method.In Step 1, the column and row differences are calculated, and the edge content, which exhibits the high-frequency components of the input images, is obtained, as shown in Figure 2. In Step 2, the row and column differences are combined with the help of the Euclidean distance, and the results for the sample images are given in Figure 3.In the third step of the method, the edge information, obtained using the differences of each pixel, is convolved with the Gaussian kernel with s = 10 in order to include the contribution of the neighbors of the relevant pixel.The GDs obtained are shown in Figure 4.In Step 4, weighting factors are obtained using GDs and visualized in Figure 5 using the jet coloring map.Here, the red color indicates that the numerical value of the weighting factor for the relevant pixel is one, which is the highest ratio, and the blue color indicates that the lowest value is zero.When the weighting factor matrix (fw1) of the visible image is examined, the outer edges of the lips, nose, and eyes are enhanced.On the other hand, when the weighting factor matrix (fw2) of the near-infrared image is examined, details such as the iris and nostrils seem to have higher factors.The fused image (F), obtained in the fifth step of the method with the weighted average using the weighting factors, is given in Figure 6.When the final fused image is examined, it can be seen that the details that are present in the visible image but not in the infrared image, and vice versa, are combined into a single composite image.

Optimization of GD Parameters
A Gaussian filter is defined by two parameters, as given in Equation ( 4): the size of the filter (s) and the standard deviation of the Gaussian distribution .Using predefined values for s and  may not be suitable for all images.Therefore, an optimal approach to determine the best parameter set for any input image is proposed in this section.
A block diagram of the proposed optimal scheme is illustrated in Figure 8.As can be seen in the figure, pattern search (PS) is chosen as the optimizer due to its simplicity and robustness.Also, PS is a well-known, derivative-free algorithm that does not require a gradient [55].The steps of the proposed Gaussian of differences with pattern search (GDPS) method can be summarized as follows: 1. Define the maximum iteration number of PS and set the initial values of GD parameters.2. Evaluate the initial solution and calculate its fitness value (overall quality of the fused image):   The fusion results are promising, as shown in the visual steps of the proposed GD method.In Step 1, the column and row differences are calculated, and the edge content, which exhibits the high-frequency components of the input images, is obtained, as shown in Figure 2. In Step 2, the row and column differences are combined with the help of the Euclidean distance, and the results for the sample images are given in Figure 3.In the third step of the method, the edge information, obtained using the differences of each pixel, is convolved with the Gaussian kernel with s = 10 in order to include the contribution of the neighbors of the relevant pixel.The GDs obtained are shown in Figure 4.In Step 4, weighting factors are obtained using GDs and visualized in Figure 5 using the jet coloring map.Here, the red color indicates that the numerical value of the weighting factor for the relevant pixel is one, which is the highest ratio, and the blue color indicates that the lowest value is zero.When the weighting factor matrix (fw1) of the visible image is examined, the outer edges of the lips, nose, and eyes are enhanced.On the other hand, when the weighting factor matrix (fw2) of the near-infrared image is examined, details such as the iris and nostrils seem to have higher factors.The fused image (F), obtained in the fifth step of the method with the weighted average using the weighting factors, is given in Figure 6.When the final fused image is examined, it can be seen that the details that are present in the visible image but not in the infrared image, and vice versa, are combined into a single composite image.

Optimization of GD Parameters
A Gaussian filter is defined by two parameters, as given in Equation ( 4): the size of the filter (s) and the standard deviation of the Gaussian distribution .Using predefined values for s and  may not be suitable for all images.Therefore, an optimal approach to determine the best parameter set for any input image is proposed in this section.
A block diagram of the proposed optimal scheme is illustrated in Figure 8.As can be seen in the figure, pattern search (PS) is chosen as the optimizer due to its simplicity and robustness.Also, PS is a well-known, derivative-free algorithm that does not require a gradient [55].The steps of the proposed Gaussian of differences with pattern search (GDPS) method can be summarized as follows: 1. Define the maximum iteration number of PS and set the initial values of GD parameters.2. Evaluate the initial solution and calculate its fitness value (overall quality of the fused image): The fusion results are promising, as shown in the visual steps of the proposed GD method.In Step 1, the column and row differences are calculated, and the edge content, which exhibits the high-frequency components of the input images, is obtained, as shown in Figure 2. In Step 2, the row and column differences are combined with the help of the Euclidean distance, and the results for the sample images are given in Figure 3.In the third step of the method, the edge information, obtained using the differences of each pixel, is convolved with the Gaussian kernel with s = 10 in order to include the contribution of the neighbors of the relevant pixel.The GDs obtained are shown in Figure 4.In Step 4, weighting factors are obtained using GDs and visualized in Figure 5 using the jet coloring map.Here, the red color indicates that the numerical value of the weighting factor for the relevant pixel is one, which is the highest ratio, and the blue color indicates that the lowest value is zero.When the weighting factor matrix (fw 1 ) of the visible image is examined, the outer edges of the lips, nose, and eyes are enhanced.On the other hand, when the weighting factor matrix (fw 2 ) of the near-infrared image is examined, details such as the iris and nostrils seem to have higher factors.The fused image (F), obtained in the fifth step of the method with the weighted average using the weighting factors, is given in Figure 6.When the final fused image is examined, it can be seen that the details that are present in the visible image but not in the infrared image, and vice versa, are combined into a single composite image.

Optimization of GD Parameters
A Gaussian filter is defined by two parameters, as given in Equation ( 4): the size of the filter (s) and the standard deviation of the Gaussian distribution σ.Using predefined values for s and σ may not be suitable for all images.Therefore, an optimal approach to determine the best parameter set for any input image is proposed in this section.
A block diagram of the proposed optimal scheme is illustrated in Figure 8.As can be seen in the figure, pattern search (PS) is chosen as the optimizer due to its simplicity and robustness.Also, PS is a well-known, derivative-free algorithm that does not require a gradient [55].The steps of the proposed Gaussian of differences with pattern search (GDPS) method can be summarized as follows: 1.
Define the maximum iteration number of PS and set the initial values of GD parameters.

2.
Evaluate the initial solution and calculate its fitness value (overall quality of the fused image): a. Apply all steps of the proposed GD fusion method explained in the previous section (Equations ( 1)-( 6)).b.
Calculate the fused image quality using an image metric (see Section 3.3).
where Q is the image quality metric to be maximized, F is the fused image, s is the size of the Gaussian filter, and σ is the standard deviation of the Gaussian distribution.

3.
Apply the operators of PS to find a better GD parameter solution that maximizes the fused image quality.4.
Repeat Steps 2 and 3 until the maximum iteration number or a predefined stopping condition is reached.
ntropy 2023, 25, x FOR PEER REVIEW 8 of 33 a. Apply all steps of the proposed GD fusion method explained in the previous section (Equations ( 1)-( 6)).b.Calculate the fused image quality using an image metric (see Section 3.3). = ((, )) (7) where Q is the image quality metric to be maximized, F is the fused image, s is the size of the Gaussian filter, and  is the standard deviation of the Gaussian distribution.
3. Apply the operators of PS to find a better GD parameter solution that maximizes the fused image quality.4. Repeat Steps 2 and 3 until the maximum iteration number or a predefined stopping condition is reached.

Experimental Results
For this section, a comprehensive series of experiments were conducted to assess the performance of the proposed GD method.As explained in Section 2, the GD method has only two control parameters: the size of the Gaussian kernel (s) and the standard deviation of the Gaussian distribution ().In the experiments, two types of cases were evaluated:

•
First, a predefined parameter set for GD was used.s values of 5, 10, and 15 values, named GD5, GD10, and GD15, respectively, were evaluated.In this case, the second parameter  was defined according to the value of the filter size,  = /3.

•
Second, the parameters of GD were adaptively determined by using the pattern search optimization algorithm to maximize the image quality.Unreported intensive experiments have shown that using Qabf, Qcb, and Qcv as fitness functions generates the best results.Therefore, the versions of this case were named GDPSQABF, GDP-SQCB, and GDPSQCV, respectively.

Image Dataset
To validate the performance of the proposed GD method, four different types of image fusion cases were selected: multi-modal medical images [56], multi-sensor infrared and visible images [45], multi-focus images [57], and multi-exposure images [58].The specifications of the images used the experiments are summarized in Table 1.

Experimental Results
For this section, a comprehensive series of experiments were conducted to assess the performance of the proposed GD method.As explained in Section 2, the GD method has only two control parameters: the size of the Gaussian kernel (s) and the standard deviation of the Gaussian distribution (σ).In the experiments, two types of cases were evaluated:

•
First, a predefined parameter set for GD was used.s values of 5, 10, and 15 values, named GD5, GD10, and GD15, respectively, were evaluated.In this case, the second parameter σ was defined according to the value of the filter size, σ = s/3.

•
Second, the parameters of GD were adaptively determined by using the pattern search optimization algorithm to maximize the image quality.Unreported intensive experiments have shown that using Qabf, Q cb , and Q cv as fitness functions generates the best results.Therefore, the versions of this case were named GDPSQABF, GDPSQCB, and GDPSQCV, respectively.

Image Dataset
To validate the performance of the proposed GD method, four different types of image fusion cases were selected: multi-modal medical images [56], multi-sensor infrared and visible images [45], multi-focus images [57], and multi-exposure images [58].The specifications of the images used the experiments are summarized in Table 1.The multi-modal medical image dataset had eight pairs of images, which are shown in Figure 9.The multi-sensor infrared and visible image dataset had 14 pairs of images, which are shown in Figure 10.The multi-focus dataset had 20 pairs of images, which are shown in Figure 11.And the multi-exposure image dataset had six pairs of images, which are shown in Figure 12.
, x FOR PEER REVIEW 9 of 33 The multi-modal medical image dataset had eight pairs of images, which are shown in Figure 9.The multi-sensor infrared and visible image dataset had 14 pairs of images, which are shown in Figure 10.The multi-focus dataset had 20 pairs of images, which are shown in Figure 11.And the multi-exposure image dataset had six pairs of images, which are shown in Figure 12.

Experimental Setup
The environmental features of the experiments are summarized in Table 2. Since there is no training phase in the proposed method, a standard workstation could be sufficient.In the experiments, the MATLAB library developed by Zhang et al., published openly on GitHub, was used [59].The configuration parameters of the fusion methods used in the experiments for comparison are summarized in Table 3.For the comparison methods, the default parameters of the original authors were used.For the proposed GD method, the parameters were determined by trial and error.Therefore, six different cases of the proposed GD method were included in the experiments (GD5, GD10, GD15, GDPSQABF, GDPSDQCB, and GDPSQCV) to emphasize the stability and adaptability of our method.

Experimental Setup
The environmental features of the experiments are summarized in Table 2. Since there is no training phase in the proposed method, a standard workstation could be sufficient.
In the experiments, the MATLAB library developed by Zhang et al., published openly on GitHub, was used [59].The configuration parameters of the fusion methods used in the experiments for comparison are summarized in Table 3.For the comparison methods, the default parameters of the original authors were used.For the proposed GD method, the parameters were determined by trial and error.Therefore, six different cases of the proposed GD method were included in the experiments (GD5, GD10, GD15, GDPSQABF, GDPSDQCB, and GDPSQCV) to emphasize the stability and adaptability of our method.
The experiments were conducted on 48 pairs of images.However, due to lack of space, only eight image pairs were selected to be visualized and compared in detail in the following sections.To investigate all results, please see the Supplementary Materials section at the end of the paper.

Objective Quality Metrics
Except for the visual analysis of the fusion results, objective quality metrics were utilized to compare the proposed method with other methods quantitatively [60].The evaluation of a fused image by visual inspection included steps such as assessing the clarity and sharpness of the output image and identifying the amount of information transferred from input images to the source image.Visual evaluation is a very helpful method for comparing performances; however, visual interpretation is highly subjective.In order to make a fair comparison, the following image quality criterions were used in the experiments: Entropy (EN) is a metric that is used the measure the information content of an image [61]: where L is the number of gray levels and h I f (i) is the normalized histogram of the fused image.Mutual information (MI) is a numerical metric that measures the interdependence of two variables.It is used to measure the amount of information shared by two images.The MI for two discrete random variables U and V is defined by [62]: where p(u, v) indicates the probability density function of U and V, and p(u) and p(v) are the marginal probability density functions of U and V, respectively.The peak signal-to-noise ratio (PSNR) represents the logarithmic decibel scale ratio between the maximum potential power of a signal and the power of the noise that introduces distortion to said signal.A high PSNR value indicates high image quality.L is the number of colors in the gray level and is taken as 255 [63]: Edge-based similarity (Qabf) is obtained by weighting the normalized edge information of both source images [64]: The structure similarity index method (SSIM) is a metric with the purpose of measuring how much of the structure of the input image is preserved in the fused image [65]: The Chen-Blum metric (Q cb ) is a referenceless image quality metric inspired by human perception [66].The Q cb value is obtained by calculating the average value of the global quality map: Cross entropy (CE) serves as a metric to assess the congruity of the information content between the input images and the fused image.Reference and fused images including the same information will have a low CE value [67]: Root mean square error (RMSE) is a measure of accuracy used to realize differences in estimation errors from different estimators for a variable and is desired to be as low as possible [63]: Chen Varshney (Q cv ) is a quality metric used in image fusion based on regional information inspired by human perception [68].The lower the Q cv , the better the fusion result: where X = [X 1 , X 2 . .., X N ] input images and X F is the fused image.
For the EN, MI, PSNR, Qabf, SSIM, and Q cb metrics, higher values indicate better results.And for CE, RMSE, and Q cv , lower values indicate good performance.In the following tables, the best result is colored in green, second-best result is colored in dark red, and the third-best result is indicated by a blue color.

Medical Image Fusion
For this sub-section, medical images M#2 and M#5, shown in Figure 9, were selected from eight candidates among the dataset and tested.The visual fusion results of image set M#2 are given in Figure 13.Input Image A is a computed tomography (CT) slice image of the human brain, and Image B is a magnetic resonance (MR) image of the same section.In an ideal case, the bright bone features shown in the CT image and the tissue features shown in the MR image should be included in the fused image.As can be seen from the visual results, the GFCE image has obvious noise in the background.The FPDE and MSVD images lack contrast.The IFEVIP and VSMWSL images resemble mostly Input A (CT) and ignore Input B (MR).As a result, the ADF, CBF, GTF, HMSD, and proposed GD methods show better visual performance than others.In Table 4, the numerical results of the quality metrics of the comparison methods for M#2 are given.As can be seen in the table, the VSMWLS, proposed GD15, and proposed GDPSQCV methods show better performance according to the numerical metrics.On the other hand, GFGC, ADF, and IFEVIP show the worst performance compared to the others.
The results of the image set M#5 are given in Figure 14.As can be seen from the results, ADF, FPDE, GFCE and MSVD show poor visual performance.On the other hand, the CBF, VSMWLS, and proposed GD methods show better visual performance than other techniques.
In Table 5, the numerical results of the quality metrics of the comparison methods for M#5 are given.As can be seen in Table 5, the CNN, proposed GD10, and proposed GDPSQCV methods show better performance according to the numerical results.On the other hand, MSVD, FPDE, and GFCE show the worst performance compared to the others.

Infrared and Visible Image Fusion
Infrared images acquired at wavelengths of 750 nm-1 mm reveal the thermal radiation of objects in a scene.On the other hand, RGB color images are captured at 400 nm-750 nm wavelengths, a range which is called the visible spectrum.For this sub-section, infrared and visible images IV#4 and IV#5, shown in Figure 10, from 14 candidates among the dataset were selected and tested.The visual fusion results of image set IV#4 are given in Figure 15.Input Image A is an infrared image of a scene that depicts three people, with a gun being held by the person on the right.Image B is a visible image of the same scene.Ideally, both thermal and visible features should be included in the fused image.As can be seen from the visual results, the contrast of the GFCE image is saturated.The result of the GTF method is blurry and includes very few features from the visible image input.The result of the MSVD method has low contrast.On the other hand, the CBF, ADF, VSMWSL, CNN, and proposed GDPS methods show better performance than the others.

Infrared and Visible Image Fusion
Infrared images acquired at wavelengths of 750 nm-1 mm reveal the thermal radiation of objects in a scene.On the other hand, RGB color images are captured at 400 nm-750 nm wavelengths, a range which is called the visible spectrum.For this sub-section, infrared and visible images IV#4 and IV#5, shown in Figure 10, from 14 candidates among the dataset were selected and tested.The visual fusion results of image set IV#4 are given in Figure 15.Input Image A is an infrared image of a scene that depicts three people, with a gun being held by the person on the right.Image B is a visible image of the same scene.Ideally, both thermal and visible features should be included in the fused image.As can be seen from the visual results, the contrast of the GFCE image is saturated.The result of the GTF method is blurry and includes very few features from the visible image input.The result of the MSVD method has low contrast.On the other hand, the CBF, ADF, VSMWSL, CNN, and proposed GDPS methods show better performance than the others.From Table 6, it can be seen that CBF, VSMWSL, and the proposed GD15 and GDP-SQCB methods show better performance according to the objective metrics.On the other hand, GFCE, GTF, and MSVD show the worst performance compared to the other methods.
The results of image set IV#5 are given in Figure 16.As can be seen from the results, CBF, GTF, and all of the GD methods except GDPSQCB show poor visual performance.On the other hand, the HMSD and MSVD methods show better visual performance than the other techniques.
In Table 7, the quantitative fusion results are given.As can be seen, HMSD, MSVD, FPDE, and GDPSQABF show better performance according to the objective metrics.On the other hand, GFCE, GTF, and the proposed GD5, GD10, GD15, and GDPSQCV methods show the worst performance compared to the other methods.

Multi-Focus Image Fusion
Images captured using a single lens of scenes containing objects at different distances have blurry regions.To extend the depth of field, images with different focal lengths are fused.
For this sub-section, multi-focus images F#11 and F#15, shown in Figure 11, from 20 candidates among the dataset were selected and tested.In Figure 17, the fusion results Entropy 2023, 25, 1215 20 of 31 of test image F#11 are given.In Input Image A, the near objects (hand and camera) are in focus, while in Input Image B, the far object (globe) is in focus.An everywhere-in-focus image is desired, which the fused image provides.
The visual results show that the contrasts of the GFCE and IFEVIP images are saturated.The GTF result is blurry (hand and camera).The MSVD, ADF, and FPDE results are also not sharp (globe).On the other hand, CBF, HMSD, VSMWSL, CNN, and the proposed GDPS methods show better performance than the others.In Table 8, the numerical results of the quality metrics of the comparison methods for F#11 are given.As can be seen in the table, CBF, CNN, and the proposed GD15, GD10, GDPSQCV, and GDPSQCB methods show better performance according to the numerical results.On the other hand, GFCE, IFEVIP, and MSVD show the worst performance compared to the others.The results of image set F#15 are given in Figure 18.As can be seen from the results, IFEVIP and GFCE show very poor visual performance.The results of MSVD and GTF contain blurry regions.On the other hand, CBF, VSMWLS, HMSD, ADF, CNN, and the proposed GDPSQCB methods show better visual performance than the other techniques.
From Table 9, it can be seen that GTF, CBF, CNN, and the proposed GDPSQCB, GD15, and GD10 methods show better performance according to the objective metrics.On the other hand, GFCE, IFEVIP, and MSVD show the worst performance compared to other methods.

Multi-Exposure Image Fusion
In the last case, image fusion algorithms were compared with regard to their use on multi-exposure images selected from six candidates among the dataset (images E#5 and E#6 of Figure 12).For a first example, the visual results of image E#5 are given in Figure 19.In Input Image A, the inside of the oven is visible, and the remaining objects are saturated.However, in Input Image B, the background details are in good contrast.Multi-exposure image fusion helps us create a high-dynamic-range image in which whole regions have balanced contrast.As can be seen from the results, CBF, HMSD, VSMWLS, CNN, and the proposed GD methods exhibit good visual performance.Moreover, the IFEVIP, GFCE, and GTF methods show poorer visual performance than the other techniques.In Table 10, the numerical results of the quality metrics of the comparison methods are given for image set E#5.As can be seen in the table, ADF, FPDE, and the proposed GD15 and GDPSQCV methods show better performance according to the numerical results.On the other hand, GFCE, IFEVIP, and GTF show the worst performance compared to the others.
The results of image set E#6 are given in Figure 20.As can be seen from the results, CBF, GTF, and GD5 show poor visual performance.Otherwise, GFCE, VSMWLS, HMSD, ADF, CNN, and the proposed GDPSQCV method show better visual performance than the other techniques.In Table 11, the quantitative results of the comparison methods are given for image set E#6.As can be seen in the table, ADF, FPDE, and GDPSQCV show better performance according to the numerical results.On the other hand, GFCE, IFEVIP, and GTF show the worst performance compared to the others.

Overall Comparison
To evaluate the numerical results more easily, the average rankings of the methods with regards to all of the quality metrics were calculated for all 48 images used in the experiments.The best ranking was set to first, and the worst ranking was set to sixteenth according to the quality metric value of each method, as we have sixteen methods in total.Each fusion application type is given in a separate table.
Table 12 shows the ranking of each method for the fusion of multi-modal medical images, including M#1 to M#8.At the bottom of the table, the average ranking of each method compared to all of the images for medical image fusion is indicated.As can be seen in Table 12, overall better results in average ranking were obtained with GD10, GD15, and GDPSQCB, whose average ranking was around sixth.GFCE and MSVD were the two worst methods with an average ranking of ~12th.
Table 13 shows the ranking of each method for the fusion of infrared and visible images, including IV#1 to IV#14.As can be seen in Table 13, overall better average rankings were obtained with HMSD, GDPSQCV, GDPSQABF, and CNN, whose average ranking was around seventh.~GTF was the worst method an average ranking of ~11th.
The ranking of each method for the fusion of multi-focus images, including F#1 to F#20, are given in Table 14.As can be seen from the results, overall better average rankings were obtained with GD15, GDPSQCV, GD10, CBF, and CNN, whose average ranking was around sixth.GFCE and IFEVIP were the worst methods with an average ranking of ~14th average ranking.
The ranking of each method for the fusion of multi-exposure images, including E#1 to E#6, are given in Table 15.As can be seen from the results, overall better average rankings were obtained with GDPSQCV, GDPSQABF, and ADF, whose average ranking was around fifth.GFCE was the worst method with an average ranking of ~13th.
The global average rankings and average CPU time consumptions of the methods for all 48 images are given in Table 16.As can be seen from the table, the proposed GD methods take the first three best rankings.The methods can be ordered from best to worst as GDPSQCV, GD15, GDPSQABF, GDPSQCB, GD10, HMSD, CNN, VSMWLS, ADF, FPDE, GD5, CBF, MSVD, GTF, IFEVIP, and GFCE.Table 16 also shows the global average CPU time consumptions of the methods in seconds.The execution time of an image processing method is directly affected by its complexity and the CPU capacity it is run on, as shown in [69]: the lower the CPU time, the faster the execution time of the method.According to the numerical results, IFEVIP, GD5, and GD10 are the fastest methods compared to the others.In total, 48 pairs of test images were used in the experiments.However, only two pairs of test images were detailed and visualized for each of the four different types of image fusion in the experiments.The fusion results of all images in the dataset can be found at the Supplementary Materials section.In addition to visual subjective evaluations, nine objective quality metrics were utilized to compare the proposed GD method with other fusion methods.
Extensive experiments have shown that the proposed GDPSQCV method attained an average rank of 6.44th among 16 methods, when considering all quality metrics and all test images, which is the best ranking of all of the methods.Moreover, the average CPU consumption time of GD15, which is the second best in overall ranking, is about 0.20 s, which is only 0.05 s slower than IFEVIP (revealed as the fastest method in the experiments).However, it must be noted that IFEVIPs average ranking is 11.41th.In addition to this, the proposed GD15 is ~115× faster than the CNN method in terms of average CPU consumption time for the fusion of 48 image pairs on an Intel i7 CPU clocked @ 4 GHz PC without parallel programming.Increasing the Gaussian filter size increases the success of the proposed method.Namely, GD15 obtained better results than GD10, and GD10 obtained better results than GD5.However, unreported experiments showed that increasing the filter size causes undesirable visual effects on the fused image.Optimal versions of GD have better performance compared to their non-adaptive versions such as GD5, GD10, and GD15.However, the CPU computing times of GDPS versions are much higher.
The main limitation of the proposed method is that it does not guarantee the best result in a particular application.However, it is capable of being a general fusion scheme and gives better results in average for any kind of fusion application.In future studies, optimization algorithm and the fitness function to be optimized may be improved.Metaheuristic algorithms are very promising, and multi-objective versions can improve the overall performance by optimizing two or more quality metrics together.In addition to this, GPU computing techniques may be utilized to speed up the optimization process.As a result, although it may not achieve the overall best result in all tests, the proposed GD method can be used as a simple and effective general image fusion method.

Supplementary Materials:
The following supporting information can be downloaded at: https://github.com/rifatkurban/GDfusion, fused images and numerical results of input image pairs in the dataset.

Figure 1 .
Figure 1.Proposed general image fusion method based on pixel-based linear weighting usin Gaussian of differences (GD).

Figure 1 .
Figure 1.Proposed general image fusion method based on pixel-based linear weighting using the Gaussian of differences (GD).

Figure 4 .
Figure 4. Gaussian of differences (GD) of the input images.

Figure 5 .
Figure 5. Weighting factors (fw) for the input images.

Figure 5 .
Figure 5. Weighting factors (fw) for the input images.Figure 5. Weighting factors (fw) for the input images.

Figure 8 .
Figure 8. Optimization of the parameters of the proposed GD fusion method.

Figure 8 .
Figure 8. Optimization of the parameters of the proposed GD fusion method.

Figure 9 .
Figure 9. Multi-modal medical images used in the experiments.

Figure 9 .
Figure 9. Multi-modal medical images used in the experiments.

Figure 10 .
Figure 10.Multi-sensor infrared and visible images used in the experiments.

Figure 11 .
Figure 11.Multi-focus images used in the experiments.

Figure 10 . 33 Figure 10 .
Figure 10.Multi-sensor infrared and visible images used in the experiments.

Figure 11 .
Figure 11.Multi-focus images used in the experiments.Figure 11.Multi-focus images used in the experiments.

Figure 11 .
Figure 11.Multi-focus images used in the experiments.Figure 11.Multi-focus images used in the experiments.

Figure 12 .
Figure 12.Multi-exposure images used in the experiments.

Figure 12 .
Figure 12.Multi-exposure images used in the experiments.

Figure 13 .
Figure 13.Medical image set M#2 (Image A and B) and their fusion image results, obtained using comparison methods.

Figure 13 .
Figure 13.Medical image set M#2 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 14 .
Figure 14.Medical image set M#5 (Image A and B) and their fusion image results, obtained using comparison methods.Figure 14.Medical image set M#5 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 14 .
Figure 14.Medical image set M#5 (Image A and B) and their fusion image results, obtained using comparison methods.Figure 14.Medical image set M#5 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 15 .
Figure 15.Infrared and visible image set IV#4 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 16 .
Figure 16.Infrared and visible image set IV#5 (Image A and B) and their fusion image results, obtained using comparison methods.

Figure 16 .
Figure 16.Infrared and visible image set IV#5 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 17 .
Figure 17.Multi-focus image set F#11 (Image A and B) and their fusion image results, obtained using comparison methods.

Figure 17 .
Figure 17.Multi-focus image set F#11 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 18 .
Figure 18.Multi-focus image set F#15 (Image A and B) and their fusion image results, obtained using comparison methods.Figure 18. Multi-focus image set F#15 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 18 .
Figure 18.Multi-focus image set F#15 (Image A and B) and their fusion image results, obtained using comparison methods.Figure 18. Multi-focus image set F#15 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 19 .
Figure 19.Multi-exposure image set E#5 (Image A and B) and their fusion image results, obtained using comparison methods.

Figure 19 .
Figure 19.Multi-exposure image set E#5 (Images A and B) and their fusion image results, obtained using comparison methods.

Figure 20 .
Figure 20.Multi-exposure image set E#6 (Image A and B) and their fusion image results, obtained using comparison methods.

Figure 20 .
Figure 20.Multi-exposure image set E#6 (Images A and B) and their fusion image results, obtained using comparison methods.

Table 1 .
Specifications of the image dataset used in the experiments.

Table 1 .
Specifications of the image dataset used in the experiments.

Table 2 .
Specifications of the implemented environment for experiments.

Table 3 .
Configuration parameters of the fusion methods used in the experiments.

Table 2 .
Specifications of the implemented environment for experiments.

Table 3 .
Configuration parameters of the fusion methods used in the experiments.

Table 4 .
Quality metric scores of medical images set M#2, obtained using comparison methods.

Table 5 .
Quality metric scores of medical images set M#5, obtained using comparison methods.

Table 6 .
Quality metric scores of infrared and visible images set IV#4, obtained using comparison methods.

Table 7 .
Quality metric scores of infrared and visible images set IV#5, obtained using comparison methods.

Table 8 .
Quality metric scores of multi-focus images set F#11, obtained using comparison methods.

Table 9 .
Quality metric scores of multi-focus images set F#15, obtained using comparison methods.

Table 10 .
Quality metric scores of multi-exposure images set E#5, obtained using comparison methods.

Table 11 .
Quality metric scores of multi-exposure images set E#6, obtained using comparison methods.

Table 12 .
Average rankings of the methods with regard to their quality metrics for multi-modal medical images.