Multi-Focus Image Fusion Based on Fractal Dimension and Parameter Adaptive Unit-Linking Dual-Channel PCNN in Curvelet Transform Domain

Abstract

Multi-focus image fusion is an important method for obtaining fully focused information. In this paper, a novel multi-focus image fusion method based on fractal dimension (FD) and parameter adaptive unit-linking dual-channel pulse-coupled neural network (PAUDPCNN) in the curvelet transform (CVT) domain is proposed. The source images are decomposed into low-frequency and high-frequency sub-bands by CVT, respectively. The FD and PAUDPCNN models, along with consistency verification, are employed to fuse the high-frequency sub-bands, the average method is used to fuse the low-frequency sub-band, and the final fused image is generated by inverse CVT. The experimental results demonstrate that the proposed method shows superior performance in multi-focus image fusion on Lytro, MFFW, and MFI-WHU datasets.

1. Introduction

Multi-focus image fusion (MFIF) is a technique used in image processing to combine multiple images of the same scene, each taken with a different focus setting, into a single image that contains all the focused details from various depths. This method is particularly useful in scenarios where a single image captured with one focus setting cannot provide sufficient clarity for objects at different distances [1].

In practical terms, a camera or imaging device typically focuses on one specific plane, leading to blurred details for objects either closer or farther away from the focus point. By using multi-focus image fusion, multiple images taken with different focal distances are merged, resulting in a final image where all parts of the scene are in sharp focus [2]. This technique has numerous applications in fields such as medical imaging, remote sensing, microscopy, and computer vision, where it is essential to capture fine details at various depths in a single, comprehensive image.

The fusion process often involves various algorithms and techniques, including wavelet transforms, pyramid decompositions, and gradient-based methods, to selectively combine the sharpest details from each input image. These methods aim to preserve the high-frequency details while minimizing artifacts such as ghosting or blurring [3,4,5,6,7,8]. In recent years, deep learning approaches have also been integrated into multi-focus image fusion [9,10] and remote sensing image fusion [11,12], further enhancing the quality and efficiency of the fusion process.

In transform domain-based methods, the curvelet [13], contourlet [14,15], and shearlet [16,17] are commonly used. These algorithms are often applied in fields such as image enhancement [18,19,20], image denoising [21,22,23], and image fusion [24,25,26,27,28,29,30]. Vishwakarma et al. [31] introduced the multi-sensor image fusion method based on the curvelet transform and Karhunen–Loeve transform, and the algorithm has achieved good results in the fusion of multi-focus images, medical images, and infrared and visible light images, demonstrating strong robustness. Li et al. [32] introduced the MFIF method based on fractal dimension and coupled neural P systems in a nonsubsampled contourlet transform (NSCT) domain, and the algorithm was experimentally validated on three multi-focus datasets, demonstrating its effectiveness in image fusion and achieving state-of-the-art (SOTA) results. Lv et al. [33] proposed the MFIF method based on parameter-adaptive pulse-coupled neural network and fractal dimension in nonsubsampled shearlet transform (NSST) domain; in addition to achieving advantages in multi-focus image fusion, the algorithm has also shown significant results in medical, infrared and visible light image fusion.

With the rise in deep learning, several advanced fusion techniques based on neural networks have been proposed [34,35]. These methods aim to learn optimal fusion strategies from large datasets without requiring manual feature extraction or explicit fusion rules. The convolutional neural networks (CNNs) [36], generative adversarial networks (GANs) [37], autoencoders [38], transformers [39], mamba [40], and diffusion model [41] are widely used in image fusion. CNNs are trained to learn how to fuse images by automatically selecting and combining the most relevant features from multiple inputs [36]. GANs can be used to generate high-quality fused images by training a generator to combine focus information while using a discriminator to evaluate the quality of the fusion [37]. Autoencoders can be used to extract and fuse relevant features from images, reconstructing a final output with enhanced focus [38]. Ouyang et al. [42] proposed the MFIF method based on superpixel features generation Graph convolutional neural network and pixel-level feature reconstruction CNN; Zhang et al. [43] introduced the unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion; and Shihabudeen et al. [44] introduced an autoencoder deep residual network model for multi-focus image fusion. Wang et al. [45] proposed the unsupervised universal image fusion approach via a generative diffusion model; the algorithm was simulated for multi-focus image fusion, multi-exposure image fusion, infrared and visible image fusion, and medical image fusion, demonstrating its versatility and broad applicability. These deep learning-based fusion methods can achieve superior performance by learning from data and adapting to complex image characteristics, but they require a large amount of data for training and may be computationally expensive, and they also face challenges such as overfitting, where the model performs well on the training data but poorly on unseen data. To address this, regularization techniques and data augmentation are often employed to improve the generalization of the model. Moreover, the choice of network architecture and hyperparameters can significantly affect the performance of the fusion process. Researchers are continuously exploring novel architectures and optimization strategies to enhance the robustness and efficiency of these fusion techniques.

Pulse-coupled neural networks (PCNNs) have demonstrated significant advantages in the field of image fusion, particularly in multi-source image fusion, where it has garnered widespread attention and in-depth research [46]. Especially when PCNN and its improved models are combined with multi-scale transforms to fuse the low-frequency or high-frequency components of decomposed images, the fusion performance is significantly enhanced [47,48]. Deeply inspired by these excellent image fusion algorithms, in this paper, a novel multi-focus image fusion method based on fractal dimension (FD) and parameter adaptive unit-linking dual-channel PCNN (PAUDPCNN) in the curvelet transform domain is proposed to solve the noise and artifact issues. The FD-based expression is utilized to motivate the linking strength while fusing the high-frequency sub-bands. The algorithm is validated for its fusion effectiveness and universality on multi-focus datasets through simulation experiments on three common datasets: Lytro, MFFW, and MFI-WHU.

The remaining structure is as follows: Section 2 presents related work, Section 3 describes the proposed algorithm, Section 4 covers experiments and analysis, and Section 5 concludes the paper.

2. Related Work

2.1. Curvelet Transform

Curvelet transform (CVT) [13] is a mathematical tool designed to efficiently represent images with discontinuities along curves. Unlike traditional wavelet transforms, which are effective at capturing point singularities, the curvelet transform excels at representing smooth edges and features with more accuracy. It uses multi-scale, multi-directional decompositions that allow for better handling of anisotropic structures in images. This makes it particularly useful in fields like image denoising, compression, edge detection, and image fusion, where preserving fine details and sharp boundaries is crucial.

2.2. PAUDPCNN

The functional diagram of the parameter adaptive unit-linking dual-channel pulse-coupled neural network (PAUDPCNN) model is depicted in Figure 1, and it contains three components: the dendritic tree, pulse generator, and information fusion module. The PAUDPCNN model can be computed by the following [48]:

$$F^A\left(i,j\right)=S^A\left(i,j\right).$$

(1)

$$F^B\left(i,j\right)=S^B\left(i,j\right).$$

(2)

$$L_n\left(i,j\right)=\left\{\begin{array}{l}1 \mathrm{if} {\displaystyle \sum _{\left(k,l\right)\in \Omega \left(i,j\right)}{Y}_{n-1}\left(k,l\right)>0}\\ 0 \mathrm{else}\end{array}\right..$$

(3)

$$U_n^A\left(i,j\right)=F^A\left(i,j\right)\left(1+{\beta }^A\left(i,j\right)L_n\left(i,j\right)\right).$$

(4)

$$U_n^B\left(i,j\right)=F^B\left(i,j\right)\left(1+{\beta }^B\left(i,j\right)L_n\left(i,j\right)\right).$$

(5)

$$U_n\left(i,j\right)=e^{-{\alpha }_u}{U}_{n-1}\left(i,j\right)+\max \left(U_n^A\left(i,j\right),U_n^B\left(i,j\right)\right).$$

(6)

$$Y_n\left(i,j\right)=\left\{\begin{array}{l}1 \mathrm{if} U_n\left(i,j\right)>E_{n-1}\left(i,j\right)\\ 0 \mathrm{else}\end{array}\right..$$

(7)

$$E_n\left(i,j\right)=e^{-{\alpha }_E}{E}_{n-1}\left(i,j\right)+C_E{Y}_n\left(i,j\right).$$

(8)

where $S^A\left(i,j\right)$ and $S^B\left(i,j\right)$ show the external stimuli of the PAUDPCNN model corresponding to the ${\left(i,j\right)}^{th}$ pixel of the images A and B, respectively. The two symmetric feeding inputs $F^A\left(i,j\right)$ and $F^B\left(i,j\right)$ for the ${\left(i,j\right)}^{th}$ neuron store the external stimuli $S^A\left(i,j\right)$ and $S^B\left(i,j\right)$, respectively. The $L_n\left(i,j\right)$ shows the linking input at the $n^{th}$ iteration for the ${\left(i,j\right)}^{th}$ neuron, whereas its 8-neighborhood neurons are represented utilizing the set $\Omega \left(i,j\right)$. The $U_n^X\left(i,j\right)\left|X\in \left\{A,B\right\}\right.$ constitutes the internal state of $X$ at the ${\left(i,j\right)}^{th}$ neuron acquired after the $n^{th}$ iteration, which corresponds to the non-linear modulation of feeding and linking inputs. The $U_n\left(i,j\right)$ is the internal activity of the PAUDPCNN model after the $n^{th}$ iteration corresponding to the ${\left(i,j\right)}^{th}$ neuron, which can be employed to generate the fused image. The external output and dynamic threshold of the ${\left(i,j\right)}^{th}$ neuron after the $n^{th}$ iteration are represented using $Y_n\left(i,j\right)$ and $E_n\left(i,j\right)$, respectively. The ${\beta }^X\left(i,j\right)\left|X\in \left\{A,B\right\}\right.$ is the linking strength corresponding to $S^X\left(i,j\right)\in \left[0, 1\right]\left|X\in \left\{A,B\right\}\right.$, which is computed by the following that uses the sigmoidal membership function.

$${\beta }^X\left(i,j\right)={\displaystyle \frac{1}{1+e^{-u_X\left(i,j\right)}}}.$$

(9)

where $X\in \left\{A,B\right\}$. Here, $u_X\in \left[-6, 6\right]$ because the other values of $u_X$ either converge to 0 or 1. The value of $u_X$ depends on either $S^A\left(i,j\right)$ or $S^B\left(i,j\right)$ but not both. This ensures different linking strengths for corresponding pixels of the source images, which persist the behavior of HVS. The mechanisms to set the values of $u_X$ are described in Section 3. The ${\alpha }_u$ and ${\alpha }_E$ are the exponential decay constants, whereas $C_E$ is the amplitude of the dynamic threshold. The value of ${\alpha }_u$ for the PAUDPCNN model is computed by the following:

$${\alpha }_u=\log \left({\displaystyle \frac{1}{\sqrt{{\sigma }_{S^A}{\sigma }_{S^B}}}}\right).$$

(10)

where ${\sigma }_{S^X}\left|X\in \left\{A,B\right\}\right.$ is the standard deviation of $S^X$. Always, ${\alpha }_u>0$ which prevents the exponential growth of $U_n$ with respect to n. The values of $C_E$ and ${\alpha }_E$ are estimated using Equations (11) and (12), respectively. Here, $S^{{{\displaystyle A}}^{\prime }}$ and $S^{{{\displaystyle B}}^{\prime }}$ represent the Otsu threshold of $S^A\left(i,j\right)$ and $S^B\left(i,j\right)$, respectively.

$$C_E=e^{-{\alpha }_u}+\sqrt{{\displaystyle \frac{S_{\max }^A{S}_{\max }^B}{S^{A^{\prime }}{S}^{B^{\prime }}}}}.$$

(11)

$${\alpha }_E=\ln \left({\displaystyle \frac{C_E}{\sqrt{S^{A^{\prime }}{S}^{B^{\prime }}}\left(\frac{1-e^{-3{\alpha }_u}}{1-e^{-{\alpha }_u}}\right)+e^{-{\alpha }_u}\left(\sqrt{S_{\max }^A{S}_{\max }^B}-\sqrt{S^{A^{\prime }}{S}^{B^{\prime }}}\right)}}\right).$$

(12)

where $S_{\max }^X\left|X\in \left\{A, B\right\}\right.$ is the maximum value of $S^X$.

3. The Proposed Method

In this section, we will provide a detailed introduction to the multi-focus image fusion algorithm proposed in this paper, which can be divided into four parts: image decomposition, high-frequency fusion, low-frequency fusion, and image reconstruction.

3.1. Image Decomposition

The source images A and B are decomposed by CVT into low-frequency sub-bands $\left\{L_A,L_B\right\}$ and high-frequency sub-bands $\left\{H_A^{l,k},H_B^{l,k}\right\}$, respectively.

3.2. Fusion of High-Frequency Sub-Bands

The high-frequency part contains most of the noise and detailed information. The fractal and fractional models have extensive applications in fields such as image processing [49,50,51,52,53,54,55,56,57,58]. The fused high-frequency sub-bands generated by the PAUDPCNN model combined fractal dimension (FD) [32,48] and consistency verification [59]. For $\left\{H_A^{l,k},H_B^{l,k}\right\}$, the absolute values of the sub-bands are fed as external inputs, which is defined as follows:

$$F^A\left(i,j\right)=\left|H_A^{l,k}\left(i,j\right)\right|.$$

(13)

$$F^B\left(i,j\right)=\left|H_B^{l,k}\left(i,j\right)\right|.$$

(14)

The values of $U_0$,$Y_0$, and $E_0$ are initialized using the following:

$$U_0\left(i,j\right)=0, Y_0\left(i,j\right)=0\&E_0\left(i,j\right)=1\forall i,j.$$

(15)

where $E_0\left(i,j\right)=1$ ensures the immediate activation of neurons, which prevents unwanted void iterations.

The fractal dimension (FD) is defined as follows [48]:

$$F{D}_X\left(i,j\right)=g^{\max }-g^{\min }$$

(16)

where $F{D}_X\left(i,j\right)|X\in \left\{H_A^{l,k},H_B^{l,k}\right\}$ is used to define the linking strength, ${\beta }^X\left(i,j\right)\left|X\in \left\{H_A^{l,k}, H_B^{l,k}\right\}\right.$, for the high-frequency sub-bands. The values of ${\beta }^X\left(i,j\right)\left|X\in \left\{H_A^{l,k}, H_B^{l,k}\right\}\right.$ are computed using Equation (9), where $u_X\left(i,j\right)$ can be calculated by the following:

$$u_X\left(i,j\right)=-6+{\displaystyle \frac{12\left(F{D}_X\left(i,j\right)-F{D}_X^{\min }\right)}{F{D}_X^{\max }-F{D}_X^{\min }}}.$$

(17)

where $F{D}_X^{\min }$ and $F{D}_X^{\max }$ are the minimum and maximum values of $F{D}_X$, respectively. The values of ${\alpha }_u$,$C_E$, and ${\alpha }_E$ are estimated utilizing Equations (10)–(12), respectively. The PAUDPCNN model is run up to N iterations, and then the decision map $D^{l,k}\left(i,j\right)$ is computed by the following:

$$D_F^{l,k}\left(i,j\right)=\left\{\begin{array}{l}1if U_N^{H_A^{l,k}}\left(i,j\right)\ge U_N^{H_B^{l,k}}\left(i,j\right)\\ 0 else\end{array}\right..$$

(18)

where $U_N^{H_A^{l,k}}\left(i,j\right)$ and $U_N^{H_B^{l,k}}\left(i,j\right)$ are the internal activities of $H_A^{l,k}\left(i,j\right)$ and $H_B^{l,k}\left(i,j\right)$ at the $N^{th}$ iteration, respectively.

Consistency verification (CV), as introduced in [59], is a commonly used post-processing step in multi-focus image fusion algorithms to enhance the quality of the fused image. This process involves examining a specific region of the decision map where the central block originates from A, while the majority of surrounding blocks belong to B. In such a case, the central block should also be assigned to B. This can be achieved by applying a majority filter, such as a 3 × 3 averaging filter, as follows:

$$CV{D}_F^{l,k}\left(i,j\right)={\displaystyle \frac{1}{9}}{\displaystyle \sum _{m=-1}^{+1} {\displaystyle \sum _{n=-1}^{+1}{D}_F^{l,k}\left(i+m,j+n\right)}}.$$

(19)

where the $CV{D}_F^{l,k}\left(i,j\right)$ shows the filtered decision map, and the fused high-frequency sub-bands $H_F^{l,k}\left(i,j\right)$ are generated by the following:

$$H_F^{l,k}\left(i,j\right)=\left\{\begin{array}{l}{H}_A^{l,k}\left(i,j\right) if CV{D}^{l,k}\left(i,j\right)=1\\ H_B^{l,k}\left(i,j\right) else\end{array}\right.$$

(20)

3.3. Fusion of Low-Frequency Sub-Bands

The low-frequency part contains the background information of the image; the averaging method is a very common algorithm in image fusion and has wide applications in this field, so we obtain the fused low-frequency sub-band $L_F\left(i,j\right)$ using the averaging method, the corresponding equation is defined as follows:

$$L_F\left(i,j\right)={\displaystyle \frac{L_A\left(i,j\right)+L_B\left(i,j\right)}{2}}.$$

(21)

where $L_F\left(i,j\right)$ denotes the fused low-frequency sub-band.

3.4. Inverse CVT

The fused image F is obtained by using inverse CVT performed on $L_F\left(i,j\right)$ and $H_F^{l.k}\left(i,j\right)$.

The main steps of the proposed method can be summarized in Algorithm 1, and the flowchart of the proposed fusion method is depicted in Figure 2.

Algorithm 1: The proposed fusion method

Input: the source images: A and B Parameters: the number of CVT decomposition levels: L, the number of PAUDPCNN iterations: N Main step: Step 1: CVT decomposition For each source image $X\in \left\{A,B\right\}$ Perform CVT on $X$ to generate $\left\{L_X,H_X^{l,d}\right\}, l\in \left[1,L\right], k\in \left[1,K\left(l\right)\right]$; End Step 2: High-frequency sub-bands’ fusion For each level $l=1:L$       For each direction $k=1:K\left(l\right)$          For each source image $X\in \left\{A,B\right\}$          Initialize the PAUDPCNN model: $U_0\left(i,j\right)=0, Y_0\left(i,j\right)=0\&E_0\left(i,j\right)=1\forall i,j$;          Estimate the PAUDPCNN parameters ${\alpha }_u$, $C_E$, and ${\alpha }_E$ via Equations (10)–(12);          Compute the value of ${\beta }^X\left(i,j\right)|X\in \left\{H_A^{l,k},H_B^{l,k}\right\}$ using Equations (9), (16)–(17);                For each iterations $n=1:N$                Compute the PAUDPCNN model using Equations (3)–(8);                End          Get the decision map $D_F^{l,k}$ based on Equation (18);          Perform the majority filtering on decision map $D_F^{l,k}$ to guarantee the consistency using Equation (19);          Compute the fused high-frequency sub-bands $H_F^{l,k}$ according to Equation (20);          End       End End Step 3: Low-frequency sub-bands’ fusion For each source image $X\in \left\{A,B\right\}$       Merge $L_A$ and $L_B$ using Equation (21) to generate $L_F$ End Step 4: Inverse CVT Perform inverse CVT on $\left\{L_F,H_F^{l,k}\right\}$ to obtain $F$; Output: the fused image $F$.

4. Experimental Results

4.1. Experimental Setup

In this section, we utilize the classical Lytro [60], MFFW [61], and MFI-WHU [43] datasets for experimentation, with examples from each of the three datasets illustrated in Figure 3. The Lytro dataset contains 20 pairs of multi-focus data, the MFFW dataset contains 13 pairs of data, and the MFI-WHU dataset contains 120 pairs of data, and we randomly selected 30 pairs from the MFI-WHU dataset for testing. Additionally, we can refer to the dataset summary in the field of fusion compiled by authors from Wuhan University, which can be found at the following link: https://github.com/Linfeng-Tang/Image-Fusion, accessed on 30 November 2024. Our algorithm does not require training data and directly conducts simulation experiments on the test dataset.

The eight image fusion methods, GD [62], MFFGAN [43], LEGFF [63], U2Fusion [64], EBFSD [65], UUDFusion [45], EgeFusion [66], and MMAE [67], are used for comparison. The parameters for each method are configured as described in the original published papers to ensure a fair comparison. The metrics such as the edge-based similarity measurement $Q_{AB/F}$ [68], the Chen–Blum metric $Q_{CB}$ [69], the Chen–Varshney metric $Q_{CV}$ [69], the structural-similarity-based metric proposed by Piella $Q_E$ [69], the feature mutual information metric $Q_{FMI}$ [70], the mutual information metric $Q_{MI}$ [68], mean square error $Q_{MSE}$ [71], the nonlinear correlation information entropy $Q_{NCIE}$ [69], the normalized mutual information $Q_{NMI}$ [69], and peak signal-to-noise ratio $Q_{PSNR}$ [71] are used to evaluate the fusion results. Except for indicators $Q_{CV}$ and $Q_{MSE}$, the larger the values of the other indicators, the better the fusion performance. Qualitative and quantitative evaluations ensure consistency and accuracy in the comparison. In our method, N is set to 200.

The number of CVT decomposition levels has a significant impact on the fusion results, and we will now focus on analyzing these effects in detail. We tested seven different decomposition levels on the Lytro dataset and analyzed the impact of decomposition levels on fusion results using both subjective and objective evaluations, as shown in Figure 4 and

Table 1. The average quantitative comparison of different CVT decomposition levels evaluated on the Lytro dataset.

Levels	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{S}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{P}\mathit{S}\mathit{N}\mathit{R}}$
1	0.5846	0.6190	83.1454	0.7065	0.8860	6.1960	23.0143	0.8244	0.8284	35.0065
2	0.6635	0.6263	74.5671	0.8233	0.8916	6.2186	24.2584	0.8247	0.8296	34.7388
3	0.7162	0.6787	36.7803	0.8708	0.8980	6.5071	23.4865	0.8266	0.8671	34.8569
4	0.7285	0.7213	20.8957	0.8773	0.8988	6.7296	22.2336	0.8279	0.8964	35.0684
5	0.7260	0.7188	20.3408	0.8773	0.8989	6.7074	21.9907	0.8277	0.8932	35.1103
6	0.7192	0.7058	20.7622	0.8761	0.8987	6.5883	22.2003	0.8270	0.8773	35.0737
7	0.6931	0.6352	33.2248	0.8536	0.8976	3.4157	72.1418	0.8124	0.4536	29.6275

. From Figure 4, it can be observed that when the decomposition level is 1, the fusion result appears blurry. When the decomposition level is 2, artifacts emerge in the fusion result. When the decomposition level is 7, noise is introduced into the fusion result, along with darker regions. For decomposition levels 3, 4, 5, and 6, no significant differences are visible in the fusion results, and the outcomes are relatively favorable. To further evaluate the fusion performance, we compared the average metrics for levels 3–6 on the Lytro dataset. As indicated in Table 1, when the decomposition level is 4, six out of ten metrics achieve optimal values. Their values are 0.7285 for $Q_{AB/F}$, 0.7213 for $Q_{CB}$, 0.8773 for $Q_E$, 6.7296 for $Q_{MI}$, 0.8279 for $Q_{NCIE}$, and 0.8964 for $Q_{NMI}$, respectively. Based on a comprehensive analysis, we set the CVT decomposition level to 4.

4.2. Fusion Results and Discussion

Figure 5 shows the image fusion results with different methods on Lytro-01 data. From the results, we can denote that the GD method generated a blurry fused image and introduced more noise, such as a ghosting artifact in the left arm of the person. The fused image generated by the MFFGAN algorithm causes the color of the person’s hat region to darken. The LEGFF algorithm produces relatively good fusion results, but the colors of the hat and watch areas on the person in the fused image are too dark, which makes it difficult to observe the fine details. The U2Fusion algorithm causes some areas of the fused image to become brighter, while others become darker; for example, the brightness of the lawn has been enhanced, but the hat area becomes severely darkened, leading to a significant loss of information. The EBFSD algorithm did not produce a focused fused image, resulting in significant information loss and poor fusion quality. The UUDFusion algorithm introduced artifacts in the fused image, such as in the areas of the left arm, hat, and golf club; additionally, some regions, such as the shirt on the left arm, have excessively high brightness, leading to significant information loss; as a result, the overall fusion quality is poor. The EgeFusion algorithm has improved the extraction and preservation of texture and color information in the fused image; however, the overall fused image appears somewhat distorted, making it difficult to realistically observe the objects themselves; additionally, some areas, such as the left arm, have excessively high brightness, leading to significant information loss. The MMAE algorithm produced relatively good fusion results, achieving a full-focus image; however, the brightness of the left arm of the person is too high, resulting in some information loss. Through comparison, the fused image obtained by our algorithm achieved the best results; it not only provides a full-focus image but also ensures that the fused information is fully complementary, with balanced brightness and clarity.

Figure 6 is a line chart of the performance metrics obtained from simulation experiments on 20 data sets from the Lytro dataset using different fusion algorithms. The average performance metrics of the different algorithms are shown in Figure 6 and

Table 2. Quantitative average comparative analysis of different methods on the Lytro dataset.

	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{S}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{P}\mathit{S}\mathit{N}\mathit{R}}$
GD	0.7034	0.6115	123.5691	0.7874	0.8887	3.8521	150.1382	0.8139	0.5113	26.5742
MFFGAN	0.6642	0.6457	42.5655	0.8409	0.8915	6.0604	34.3748	0.8237	0.8047	33.5508
LEGFF	0.6810	0.6751	53.0073	0.8195	0.8937	5.6138	39.1523	0.8214	0.7473	32.6160
U2Fusion	0.6143	0.5682	97.5910	0.7835	0.8844	5.7765	59.4424	0.8221	0.7725	31.2098
EBFSD	0.5570	0.6076	159.9536	0.6191	0.8865	6.0670	49.9849	0.8245	0.8097	31.5131
UUDFusion	0.5107	0.5989	98.6773	0.6214	0.8703	4.8412	169.2212	0.8178	0.6417	25.9218
EgeFusion	0.3576	0.4034	340.4188	0.5032	0.8472	3.2191	77.8597	0.8120	0.4248	29.2757
MMAE	0.6326	0.6419	52.1538	0.7516	0.8846	5.2676	28.8726	0.8197	0.6995	33.7963
Proposed	0.7285	0.7213	20.8957	0.8773	0.8988	6.7296	22.2336	0.8279	0.8964	35.0684

. Considering that a lower value of $Q_{CV}$ indicates better performance, we take its negative value (i.e., $-Q_{CV}$) to illustrate the sub-figure of $Q_{CV}$. The horizontal axis represents the number of image pairs in the dataset, and the vertical axis represents the metric value. From Figure 6 and Table 2, we can observe that our algorithm achieves the optimal values in terms of the average of 10 evaluation metrics. Their values are 0.7285 for $Q_{AB/F}$, 0.7213 for $Q_{CB}$, 20.8957 for $Q_{CV}$, 0.8773 for $Q_E$, 0.8988 for $Q_{FMI}$, 6.7296 for $Q_{MI}$, 22.2336 for $Q_{MSE}$, 0.8279 for $Q_{NCIE}$, 0.8964 for $Q_{NMI}$, and 35.0684 for $Q_{PSNR}$, respectively. These quantitative results are consistent with the conclusions of the qualitative analysis, providing strong evidence of the fusion advantages of our algorithm on the Lytro dataset.

Figure 7 shows the image fusion results with different methods on MFFW-01 data. From the results, we can denote that the GD and MFFGAN algorithms generate artifacts in the fused images. Although LEGFF achieves full-focus results, some information is lost in the stone areas due to excessive brightness. The image produced by U2Fusion exhibits blurring. The EBFSD algorithm produces distortion and exhibits block artifacts. The fused image produced by UUDFusion is of poor quality, with information loss in the stone areas due to excessive brightness. Additionally, stripe patterns appear in the sea areas, and a significant amount of noise is introduced. The fused image produced by the EgeFusion algorithm suffers from severe information loss due to excessive sharpening. In the fused image produced by MMAE, the brightness of the stone area on the left is too dark, making it difficult to accurately capture the corresponding information. Through comparison, our algorithm produces the best fusion results, with both brightness and detailed information well preserved.

Figure 8 is a line chart of the performance metrics obtained from simulation experiments on 13 data sets from the MFFW dataset using different fusion algorithms. The average performance metrics of the different algorithms are shown in Figure 8 and

Table 3. Quantitative average comparative analysis of different methods on the MFFW dataset.

	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{S}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{P}\mathit{S}\mathit{N}\mathit{R}}$
GD	0.6279	0.5557	217.9965	0.7011	0.8730	3.6107	141.4848	0.8122	0.4986	28.1099
MFFGAN	0.5905	0.5851	138.1153	0.7557	0.8742	5.0498	82.9576	0.8179	0.7094	29.8959
LEGFF	0.6294	0.6032	172.4173	0.7386	0.8775	4.8088	69.3044	0.8169	0.6752	30.2400
U2Fusion	0.5537	0.5499	228.0064	0.7076	0.8690	4.8894	94.3035	0.8171	0.6992	29.1532
EBFSD	0.5570	0.5608	216.0796	0.6398	0.8766	4.9049	58.1101	0.8173	0.6874	31.1015
UUDFusion	0.4733	0.5435	265.7120	0.5482	0.8581	4.4343	172.8126	0.8154	0.6245	25.8858
EgeFusion	0.3517	0.4213	443.4456	0.4581	0.8380	3.3785	77.3397	0.8115	0.4685	29.3955
MMAE	0.5526	0.5781	424.9751	0.6336	0.8697	4.2843	57.8423	0.8146	0.6019	31.3580
Proposed	0.6290	0.6276	123.8589	0.8021	0.8796	5.1029	43.0355	0.8181	0.7199	32.5370

. From Figure 8 and Table 3, we can see that our algorithm achieves the optimal values for 9 out of the 10 evaluation metrics in terms of the average value, except for metric $Q_{AB/F}$. The LEGFF algorithm achieved the optimal value on metric $Q_{AB/F}$, reaching 0.6294; our algorithm achieved a value of 0.6290 on metric $Q_{AB/F}$, ranking second among all the algorithms we used. The values of our proposed algorithm on the other nine metrics are as follows: 0.6276 for $Q_{CB}$, 123.8589 for $Q_{CV}$, 0.8021 for $Q_E$, 0.8796 for $Q_{FMI}$, 5.1029 for $Q_{MI}$, 43.0355 for $Q_{MSE}$, 0.8181 for $Q_{NCIE}$, 0.7199 for $Q_{NMI}$, and 32.5370 for $Q_{PSNR}$. These quantitative results are consistent with the conclusions of the qualitative analysis, providing strong evidence of the fusion advantages of our algorithm on the MFFW dataset.

Figure 9 shows the image fusion results with different methods on MFI-WHU-01 data. From the results, we can see that the image fused using the GD algorithm is blurry, and the noise has increased, especially in areas like the wooden door; additionally, the marble region has become darker, resulting in information loss. The fusion image generated by the MFFGAN algorithm results in reduced brightness at the door and window areas, making it darker and unable to accurately capture interior details. The LEGFF algorithm produces a fully focused image, but the brightness in the window and marble areas is reduced, resulting in a darker appearance and causing some information to be lost. The U2Fusion method causes certain areas of the fused image to become darker, resulting in poor fusion quality. The EBFSD algorithm does not produce a fully focused image, and the image is blurry and distorted, leading to a significant loss of information. The UUDFusion algorithm results in a fused image with a yellowish tint, causing severe color distortion. The EgeFusion algorithm enhances and sharpens the details of the fused image, but it also introduces some degree of distortion, making it difficult to observe the true information in the image. The MMAE algorithm generates a fully focused fused image, and while it enhances the image’s texture and details, it also leads to some loss of real information. Through comparison, our algorithm achieves the optimal fusion result, preserving all the information from the two source images in the fused image and achieving the best complementary effect.

Figure 10 presents a line chart displaying the performance metrics obtained from simulation experiments on 30 datasets from the MFI-WHU dataset using various fusion algorithms. The average performance metrics for each algorithm are summarized in both Figure 10 and

Table 4. Quantitative average comparative analysis of different methods on the MFI-WHU dataset.

	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{M}\mathit{S}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$	${\mathit{Q}}_{\mathit{N}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{P}\mathit{S}\mathit{N}\mathit{R}}$
GD	0.6752	0.6301	105.0418	0.7754	0.8648	3.6940	121.7647	0.8136	0.5001	27.7818
MFFGAN	0.6427	0.6329	45.6960	0.7826	0.8684	5.6832	51.5176	0.8222	0.7709	31.6060
LEGFF	0.6190	0.6060	71.1462	0.7067	0.8692	4.8291	43.1211	0.8183	0.6555	32.2127
U2Fusion	0.5502	0.5156	119.8639	0.6970	0.8565	5.1498	71.8214	0.8194	0.6991	30.1022
EBFSD	0.5485	0.6531	63.7060	0.6558	0.8619	5.5230	53.3889	0.8215	0.7556	31.1799
UUDFusion	0.4576	0.6376	69.0701	0.5364	0.8470	4.3097	183.6179	0.8161	0.5839	25.5706
EgeFusion	0.2874	0.3277	537.7216	0.3757	0.8255	2.8055	86.4381	0.8111	0.3761	28.8418
MMAE	0.5916	0.6813	54.5921	0.6834	0.8628	4.9524	38.4703	0.8188	0.6730	32.6619
Proposed	0.7199	0.7875	23.7432	0.8429	0.8772	7.5215	20.5981	0.8350	1.0270	35.3640

. As shown in these figures, our algorithm achieves the best overall performance across 10 evaluation metrics. Their values are 0.7199 for $Q_{AB/F}$, 0.7875 for $Q_{CB}$, 23.7432 for $Q_{CV}$, 0.8429 for $Q_E$, 0.8772 for $Q_{FMI}$, 7.5215 for $Q_{MI}$, 20.5981 for $Q_{MSE}$, 0.8350 for $Q_{NCIE}$, 1.0270 for $Q_{NMI}$, and 35.3640 for $Q_{PSNR}$, respectively. These quantitative results align with the findings of the qualitative analysis, providing robust evidence of the superior fusion capabilities of our algorithm on the MFI-WHU dataset.

4.3. Application Extension

In this section, we extend the application of the proposed algorithm to the fusion of triple series images within the Lytro dataset [60] and multi-exposure image fusion [72]. The Lytro dataset comprises four groups of triple-series image data. If the data include three images, we need to first fuse two of them and then fuse the resulting image with the third one to obtain the final fused image. The fusion results of our proposed algorithm are illustrated in Figure 11. Specifically, Figure 11a–c represent the input images, while Figure 11d demonstrates the fusion outcome achieved by our algorithm. The results demonstrate that our algorithm is capable of producing fully focused fused images even when multiple source images are input, which further validates the versatility and robustness of our algorithm in the field of multi-focus image fusion.

Multi-exposure image fusion is a sophisticated technique designed to address the limitations of single-exposure imaging, particularly in scenes with high dynamic range (HDR) or complex lighting conditions [73]. By combining multiple images captured at different exposure levels, this method aims to produce a single, high-quality image that retains optimal detail in both bright and dark regions. The process leverages the strengths of each individual exposure, integrating them to enhance overall image quality, improve detail representation, and achieve a more balanced and visually appealing result. With applications spanning photography and beyond, multi-exposure image fusion has become an essential tool for overcoming the challenges posed by varying illumination and expanding the capabilities of digital imaging systems. We applied the proposed algorithm to the multi-exposure image dataset collected by Zhang [74] for simulation validation, and some of the results are shown in Figure 12. Figure 12a,b represent the underexposed and overexposed images, respectively, while Figure 12c illustrates the fusion result obtained by our algorithm. As can be seen from the results, the proposed algorithm demonstrates significant effectiveness in multi-exposure image fusion, producing a more detailed and vivid fused image.

5. Conclusions

In this paper, a novel multi-focus image fusion algorithm using FD and parameter adaptive unit-linking dual-channel pulse-coupled neural network (PAUDPCNN) in the CVT domain is proposed. The high-frequency sub-bands of decomposition are fused by FD and PAUDPCNN models, along with consistency verification; and the low-frequency sub-bands are fused by the average method. The public Lytro, MFFW, and MFI-WHU datasets are used to test, through subjective and objective evaluation analysis, the experimental results and show that our proposed multi-focus image fusion algorithm has robustness and better fusion performance compared with state-of-the-art algorithms. Additionally, it has a clear advantage in these 10 image fusion indicators: $Q_{AB/F}$, $Q_{CB}$, $Q_{CV}$, $Q_E$, $Q_{FMI}$, $Q_{MI}$, $Q_{MSE}$, $Q_{NCIE}$, $Q_{NMI}$, and $Q_{PSNR}$. The proposed method demonstrates its superiority in preserving the details and textures of the original images while effectively eliminating artifacts and enhancing the overall quality of the fused images. The algorithm’s adaptability to various imaging conditions and its ability to handle images with different focus points make it a promising solution for applications requiring high-resolution and clear images from multiple sources. The algorithm proposed in this paper requires selecting the decomposition level of CVT based on experimental results or prior experience. In future work, we plan to further investigate and optimize this selection process to enhance its adaptability and accuracy. Additionally, we will explore the extended applications of this algorithm in various domains, including change detection [75,76,77] and remote sensing image fusion [78], to assess its effectiveness and potential improvements.