PSGAN: Pseudo-Sensing GAN for Infrared and Low-Light Visible Image Fusion

Abstract

Infrared and visible image fusion, as a pivotal image processing technique, aims to integrate complementary information from heterogeneous source images into a high-quality fused representation. However, the intrinsic domain discrepancy between infrared and visible modalities, coupled with the susceptibility to low-light environments, significantly hampers the fusion process and degrades the quality of the results. To address these limitations, this paper proposes PSGAN, a triple-discriminator generative adversarial network incorporating a pseudo-sensing mechanism. Specifically, our method synergistically integrates the distributional alignment capability of generative models with a pseudo-sensing mechanism that simulates the inverse process of perceptual imaging. This approach effectively alleviates the domain discrepancy across disparate imaging physics, empowering the network to extract robust, cross-modal aligned features. Furthermore, an additional illumination discriminator is incorporated into the conventional dual-adversarial framework to guide the fused images toward achieving optimal luminance and contrast, even in extreme darkness. Finally, to comprehensively quantify the performance of fusion algorithms across multiple dimensions, this paper develops a multi-criteria evaluation framework based on Dempster–Shafer (D-S) evidence theory. Experimental results demonstrate that the PSGAN algorithm achieves superior fusion accuracy and strong generalization capability, while simultaneously providing significant performance gains for downstream detection tasks.

1. Introduction

Due to the limitations of imaging sensors and diverse environments, a single modality often fails to capture comprehensive scene representations. Image fusion addresses this by merging complementary information from heterogeneous sources. Specifically, infrared and visible image fusion (IVIF) has emerged as a pivotal technique within the domain of digital signal processing, proving essential for downstream tasks like object detection and semantic segmentation in challenging scenarios. IVIF aims to synthesize a fused signal that preserves both infrared thermal radiation and visible texture details. By doing so, it mitigates the performance degradation of optical sensors in low light and the inherent spectral noise or resolution constraints of infrared imaging.

Despite significant advancements in IVIF, two core challenges persist. First, the intrinsic domain discrepancy arising from disparate imaging physics—thermal radiation versus optical reflection—presents a major obstacle. This fundamental difference, compounded by the scarcity of ground-truth data, renders the extraction of cross-modal aligned features exceptionally difficult. Second is the sensor sensitivity to adverse environments. In real-world scenarios like low light, visible images suffer severe degradation, leading to the loss of critical textures [1].

To mitigate these challenges, various paradigms have been explored. Regarding domain alignment, CDTFusion [2] utilizes swap image translation, FreeFusion [3] introduces cross-reconstruction learning, while EMMA [4] and SII [5] leverage imaging priors and source reconstruction, respectively, to extract domain-adaptive features. However, relying solely on reconstruction constraints provides insufficient supervision to minimize feature distances in the domain space and fails to address severe environmental degradation. Generative model-based methods attempt to achieve fusion by aligning the distributions of source and fused images, but this approach often suffers from a lack of interpretability [6] and controllability [7]. In terms of handling low-light scenarios, DIVFusion achieves all-weather object saliency by deeply coupling low-light enhancement with multi-modal fusion. Despite its effectiveness, this approach underutilizes the potential of fine-grained feature mapping between the two synergistic tasks, leaving room for improvement in preserving intricate details across complex scenarios.

To address the aforementioned challenges, we propose a triple-discriminator GAN training framework incorporating a pseudo-sensing mechanism, which tackles the problem from the dual perspectives of simulated perceptual imaging and low-light visible image enhancement (LVIE). Methodologically, our approach begins with the inverse process of perception and imaging, learning the backward mapping from the fused result to the respective source modalities. As illustrated in Figure 1, by measuring the discrepancy between the pseudo-sensing outputs and the corresponding source images, the adversarial learning process imposes a strong physical consistency constraint on the generator. This forces the generator to produce cross-modal aligned fused images, effectively alleviating the domain discrepancy without requiring explicit ground-truth labels.

Figure 1. (a) Existing IVIF methods. (b) Proposed IVIF method incorporating a pseudo-sensing mechanism.

Specifically, the generator consists of a fusion module (UNW-Fuser) and a pseudo-sensing module (PSM). In UNW-Fuser, an enhanced U-Net architecture is employed to capture multi-scale features, where the encoder integrates a normalization-based attention mechanism (NAM) and wavelet convolution (WTConv) to synergistically expand the receptive field and suppress redundant feature responses. Furthermore, inspired by EMMA, the PSM—also based on a U-Net architecture—re-projects fused images back to the source domains to simulate the natural imaging process. This allows the discriminator to directly measure the discrepancy between source images and pseudo-sensing outputs. In addition, we integrate an additional illumination discriminator into the traditional dual-adversarial framework. Working in tandem with the generator, this discriminator provides refined feedback to guide the fused image toward optimal luminance, enhancing visual quality in dark scenarios. Finally, to evaluate the holistic performance of various algorithms across multiple metrics, we establish a multi-criteria evaluation framework based on Dempster–Shafer (DS) theory. Specifically, our main contributions are as follows:

1.

Our method synergistically integrates the distributional alignment capability of generative models with the physical priors of a pseudo-sensing mechanism that simulates the inverse process of perceptual imaging. This dual-constrained approach effectively alleviates the intrinsic domain discrepancy between infrared and visible modalities, enabling the network to extract cross-modal aligned features across disparate imaging physics.

2.

An enhanced multi-scale fusion module, termed UNW-Fuser, is designed. It effectively expands the effective receptive field and boosts deep feature extraction.

3.

A triple-discriminator adversarial architecture incorporating an illumination discriminator is constructed. By introducing dedicated illumination constraints, it ensures imaging quality under extreme low-light conditions.

4.

A multi-criteria evaluation framework based on DS evidence theory is developed, addressing the evaluation bias caused by limited metrics and heavy subjectivity in existing methods.

The rest of the article follows. Section 2 reviews related works. Section 3 introduces the specific structure of the module. Section 4 presents the experimental details and discusses the experimental results. Conclusions are given in Section 5.

2. Related Works

This section reviews existing IVIF methods and introduces the coupled system and pseudo-sensor technology in IVIF.

2.1. Infrared and Visible Image Fusion

Generally, IVIF methods are categorized into traditional and deep learning-based algorithms. Traditional approaches encompass multi-scale transformation [8,9], sparse representation [10], subspace-based transformation [11], saliency-based algorithms [12,13], and hybrid methods [14,15]. Driven by the rapid advancement of artificial intelligence, deep learning-based IVIF methods have witnessed significant growth [16]. Mainstream paradigms include convolutional neural networks (CNNs), autoencoder-based (AE) methods, generative adversarial networks (GANs), and Transformer-based models. Liu et al. [17] explored CNN-based fusion by deriving binary decision maps for multi-focus image fusion. Standard convolutions are limited by their local receptive fields, whereas Transformer architectures use self-attention to model long-range dependencies and aggregate contextual information. For example, DepthNet [18] combines convolutional layers with a vision Transformer to capture complementary local and global features. Ma et al. [19] introduced the GAN paradigm to IVIF, formulating fusion as adversarial min-max optimization between a generator and a discriminator. More recent methods incorporate additional priors or generative paradigms. TAFuse [20] uses a multi-scale autoencoder and frequency-domain processing to enhance texture perception and reconstruction. DiTFuse [21] adopts a Diffusion Transformer architecture to support unified, instruction-controllable image fusion across multiple tasks.

2.2. Joint Low-Light Enhancement and Image Fusion

IVIF serves as a critical bridge within multi-task processing systems, linking pre-processing stages (e.g., LVIE) with downstream high-level vision tasks [17]. In recent years, significant progress has been made in fusion methods tailored for low-light environments. LLVIP [18], as a cornerstone study in this field, not only contributed a high-quality paired low-light dataset but also established a benchmark for the coupled evaluation of image fusion and object detection. At the algorithmic level, DIVFusion [19] attempts to balance the dual tasks by introducing an illumination-aware loss; however, it still faces limitations of insufficient contrast in extreme darkness due to the lack of explicit modeling of the physical imaging process. Meanwhile, although PIA [22] utilizes illumination information to guide feature extraction, it is prone to color bias and structural distortion when dealing with non-uniform illumination degradation. These challenges fundamentally stem from the incompatibility of cross-task feature mappings. In contrast, the proposed PSGAN achieves deep decoupling and precise synergy between illumination enhancement and feature fusion through the collaboration of triple discriminators.

2.3. Pseudo-Sensing Mechanisms

Pseudo-sensing mechanisms, similar to the concept of source mapping, are learning mechanisms that simulate the imaging process of real sensors. This technique typically adopts a data-driven approach to learn a mapping function from an intermediate representation to a specific modality image, thereby enabling self-supervised optimization in scenarios where ground-truth labels are unavailable. A prominent example is the EMMA framework [4], which incorporates a dedicated pseudo-sensing module to simulate sensor imaging responses, providing physically consistent supervisory signals for the fusion network via equivariant priors. In the field of image processing, such pseudo-sensing techniques are widely applied in tasks such as cross-modal learning, image reconstruction, and image fusion.

3. Method

This section provides a detailed description of the network framework and loss functions of the proposed PSGAN.

3.1. Network Architecture

Figure 2 illustrates the proposed framework, including one generator and three discriminators. By integrating a fusion module (UNW-Fuser) and a pseudo-sensing module (PSM), the generator synergizes feature fusion with inverse imaging simulation. Specifically, the UNW-Fuser leverages NAM and WTConv for detail enhancement, while the PSM ensures physical consistency via source-domain re-projection. On the adversarial side, an independent URetinex-Net enhancement branch transforms the low-light visible image into an enhanced reference image for the illumination discriminator. Through adversarial supervision, the network is guided to generate fused images with high contrast, rich textures, and natural brightness, even under extremely low-light conditions.

Figure 2. The overall framework of the proposed three-discriminator GAN integrated with a pseudo-sensing mechanism.

3.1.1. Fusion Module

As illustrated in Figure 3, the proposed UNW-Fuser adopts an encoder–decoder structure integrated with a specialized fusion layer. The encoder employs a U-Net-based dual-path architecture to perform multi-scale feature extraction, taking infrared and visible images as parallel inputs. Two independent branches are designed to extract hierarchical representations (from shallow textures to deep semantics), thereby enhancing feature efficacy. The encoding pipeline consists of four down-sampling blocks, where the initial three blocks utilize two convolutional layers with PReLU activation, followed by max-pooling for spatial reduction. To balance efficiency and performance, $3\times 3$ kernels are predominantly employed.

Figure 3. The overall framework of the proposed fusion module integrates NAM and WTConv in the generator.

In the decoder, three up-sampling blocks are used for feature reconstruction, while skip connections facilitate the preservation of high-resolution details. Within these connections, we introduce the Normalization-based Attention Mechanism (NAM) [20]. Inspired by CBAM, NAM is a lightweight module that re-purposes the Batch Normalization (BN) scaling factor to quantify feature importance. For the channel attention sub-module, NAM leverages the trainable factor $\gamma$ from BN to measure channel variance:

$$B_{\mathrm{out}}=\gamma ·{\displaystyle \frac{B_{\mathrm{in}}-{\mu }_B}{\sqrt{{\sigma }_B^2+\epsilon }}}+\beta$$

(1)

The channel attention map $M_c$ is then derived using normalized weights $W_{\gamma }$:

$$M_c=\mathrm{sigmoid}\left(W_{\gamma }\left(\mathrm{BN}\left(F_1\right)\right)\right),W_{\gamma }={\displaystyle \frac{{\gamma }_i}{{\sum }_j{\gamma }_j}}$$

(2)

Similarly, NAM applies a scaling factor $\lambda$ to the spatial dimension for pixel normalization, yielding the spatial attention map $M_B$:

$$M_B=\mathrm{sigmoid}\left(W_{\lambda }\left({\mathrm{BN}}_s\left(F_2\right)\right)\right)$$

(3)

To enforce sparsity and suppress non-salient features, an $L_1$ regularization term is incorporated into the loss function:

$${\mathcal{L}}_{reg}=\rho \sum g\left(\gamma \right)+\rho \sum g\left(\lambda \right)$$

(4)

where $g(·)$ denotes the $L_1$ norm penalty and $\rho$ is the balancing hyperparameter.

The fusion layer hierarchically integrates the encoder-extracted thermal radiation and texture features. To expand the receptive field and optimize the fusion of low- and high-frequency components, we incorporate the Wavelet Convolution Module (WTConv) [23]. Specifically, each decomposed sub-band is processed independently by a lightweight convolutional network to capture localized features. Subsequently, an Inverse Wavelet Transform (IWT) is employed to reconstruct enhanced feature representations by synergizing frequency and spatial domain information. WTConv supports hierarchical nesting of wavelet decompositions to further augment multi-scale perception. We adopt the efficient Haar wavelet as the basis and perform a three-level cascaded decomposition, significantly improving the model’s sensitivity to low-frequency information without substantial parameter overhead.

3.1.2. Pseudo-Sensing Module

In the implementation, PSM is designed to generate pseudo-imaging results $I_{p\left(ir\right)}$ and $I_{p\left(vi\right)}$ from fused images. Directly modeling the explicit perceptual mappings $P\left(ir\right)$ and $P\left(vi\right)$ is non-trivial in practice. Therefore, inspired by the EMMA framework [4], a data-driven modeling strategy is adopted to facilitate the self-supervised learning of these inverse sensing processes. Specifically, this study selects 14 SOTA fusion methods [4,7,19,22,24,25,26,27,28,29,30,31,32,33] and utilizes their fusion outputs as pseudo-fused images to construct an approximate fusion ground-truth dataset. Considering that the fusion module already employs a U-Net structure, to maintain architectural consistency and training synergy within the overall network, we likewise adopt the U-Net architecture as the basis for the pseudo-sensing module, as illustrated in Figure 4. Two separate perceptual networks are trained to approximate the mappings from fused images to the infrared and visible domains. The associated loss functions are detailed in Section 3.2.

Figure 4. The overall framework of the proposed pseudo-sensing module.

The PSM adopts a fully decoupled training strategy, with its parameters kept frozen during the optimization of the overall PSGAN framework. This design establishes a deterministic mapping from fused images back to the source domains, ensuring physical consistency and architectural interpretability. Furthermore, PSGAN relies solely on fundamental imaging priors without requiring task-specific structural assumptions.

3.1.3. U-Retinex

To strengthen the illumination discriminator, we incorporate the URetinex-Net framework [34] as a preprocessing module to generate enhanced illumination reference images. Specifically, given the visible image $I_{vi}$, URetinex-Net produces an enhanced reference image $I_{lvi}$, formulated as

$$I_{lvi}=\text{URetinex-Net}\left(I_{vi}\right).$$

(5)

The enhanced image $I_{lvi}$ is subsequently provided to the illumination discriminator $D_{lvi}$ as a real reference sample for adversarial learning. Grounded in Retinex theory, this approach decomposes an image into reflectance and illumination components, mapping the traditional optimization process into a learnable network via a deep unfolding strategy. As illustrated in Figure 5, URetinex-Net consists of three sub-modules: (i) an initialization module for structure-preserving feature estimation; (ii) an unfolded optimization module that progressively mitigates degradation through multi-stage iterations, utilizing dual sub-networks to learn implicit priors for detail preservation; and (iii) an illumination adjustment module for flexible brightness modulation. By embedding this Retinex-based strategy into the GAN architecture, PSGAN achieves effective brightness enhancement while resolving the coupling conflicts between image enhancement and fusion tasks.

Figure 5. The overall framework of URetinex-Net.

3.1.4. Discriminator Architecture

The three discriminators adopt identical network architectures with independently learned parameters, as illustrated in Figure 2. During training, $I_{ir}$, $I_{vi}$, $I_{p\left(ir\right)}$, $I_{p\left(vi\right)}$, and $I_{lvi}$ are fed into their corresponding discriminators. Each discriminator consists of a series of $3\times 3$ convolutional layers with ReLU activation functions and batch normalization for feature extraction, followed by a fully connected output layer that produces an image-level adversarial response. The infrared discriminator encourages the fused image $I_f$ to preserve more infrared intensity information, while the visible discriminator guides the network to retain detailed textures from the visible image. Furthermore, the illumination discriminator promotes the preservation of natural brightness in the fused image. As adversarial training progresses, the generator learns to produce fused images that better satisfy the objectives of all three discriminators, thereby improving the overall fusion performance.

3.2. Loss Function

3.2.1. Generator Loss Function

The loss functions for both UNW-Fuser and PSM consist of adversarial loss, content loss, and structural similarity index measure (SSIM) loss, formulated as

$$L_{total}=L_{adv}+\lambda L_{con}$$

(6)

where the content loss $L_{con}$ is further decomposed into intensity, gradient, and SSIM components:

$$L_{con}=L_{int}+\alpha L_{grad}+\beta L_{SSIM}$$

(7)

where $\alpha$ and $\beta$ denote the weighting coefficients assigned to balance the gradient loss $L_{grad}$ and the structural similarity loss $L_{SSIM}$, respectively.

The gradient loss ${\mathcal{L}}_{grad}$ is designed to preserve texture details from the source images for sharper fusion results:

$${\mathcal{L}}_{grad}={\displaystyle \frac{1}{HW}}{∥\max \left(\right|\nabla I_{p\left(ir\right)}|,|\nabla I_{p\left(vis\right)}\left|\right)-\max \left(\right|\nabla I_{ir}|,|\nabla I_{vis}\left|\right)∥}_1$$

(8)

where H and W denote the height and width of the image. $\max \left(\right|\nabla I_{p\left(ir\right)}|,|\nabla I_{p\left(vis\right)}\left|\right)$ represents the gradient map of the pseudo-perceived images, while $\max \left(\right|\nabla I_{ir}|,|\nabla I_{vi}\left|\right)$ represents the pixel-wise maximum gradient between $I_{ir}$ and $I_{vi}$.

The intensity loss ${\mathcal{L}}_{int}$ ensures consistent intensity distribution between the fused image $I_f$ and the input images:

$${\mathcal{L}}_{int}={\displaystyle \frac{1}{HW}}{∥\max (I_{p\left(ir\right)},I_{p\left(vis\right)})-\max (I_{ir},I_{vi})∥}_1$$

(9)

The SSIM loss evaluates image quality by integrating luminance, structure, and contrast:

$$L_{SSIM}=\eta (1-\mathrm{SSIM}(I_f,I_{p\left(ir\right)}))+(1-\eta )(1-\mathrm{SSIM}(I_f,I_{p\left(vis\right)}))$$

(10)

where $\eta \in [0,1]$ is a weighting coefficient used to balance the structural contributions from the infrared and visible domains.

3.2.2. Discriminator Loss Function

Three discriminators, denoted as $D_{vi}$, $D_{ir}$, and $D_{lvie}$, are designed to ensure the perceptual realism of the fused result by making it visually indistinguishable from the source and enhanced images. The adversarial loss ${\mathcal{L}}_{adv}$ for the generator is defined as follows:

$${\mathcal{L}}_{adv}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N}\left[\log D_{vi}\left(G\left(I_{p\left(vi\right)}^n\right)\right)+\log D_{ir}\left(G\left(I_{p\left(ir\right)}^n\right)\right)+\log D_{lvie}\left(G\left(I_{lvi}^n\right)\right)\right]$$

(11)

To optimize the discriminators while ensuring training stability, we adopt the WGAN-GP framework. The loss function for each discriminator $D_k$ is formulated using a unified expression:

$$L_{D_k}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N}{\left[D_k\left(I_k^n\right)-D_k\left(I_p^n\right)+\gamma (1-\parallel \nabla D_k\left(\widehat{I}\right){\parallel }_2)\right]}^2$$

(12)

where $k\in \{vi,ir,lvie\}$ represents the visible, infrared, and illumination domains, respectively. Accordingly, $I_k$ denotes the corresponding target image ($I_{vi}$, $I_{ir}$, or $I_{lvi}$), while $I_p$ denotes the generated pseudo-result or enhanced output. Here, $\gamma$ is the regularization parameter, ${\parallel ·\parallel }_2$ is the $L_2$ norm, and ∇ denotes the gradient operator. The difference between the first two terms approximates the Wasserstein distance between the generated and target distributions, while the final term serves as the gradient penalty to regularize the discriminator’s Lipschitz continuity. We set the hyperparameters $\alpha =25$, $\lambda =10$, $\eta =0.5$, and $\gamma =10$. These values are determined by evaluating the simulated metrics under varying parameter configurations to ensure optimal performance.

4. Experimental Results and Analysis

This section first introduces the overall experimental setup and implementation details. Subsequently, a series of generalization experiments are conducted to validate the performance advantages of the proposed method across diverse datasets. Then, ablation studies are performed to evaluate the contribution of each key module to the overall performance. Finally, downstream object detection experiments are designed to further demonstrate the effectiveness and applicability of the proposed approach in real-world visual applications.

4.1. Configurations and Implementation Details

Datasets: Extensive experiments were conducted on the MSRS [22], M3FD [35], TNO [36] and LLVIP [18] datasets. Comparison of algorithms: Eleven state-of-the-art (SOTA) algorithms are selected for qualitative and quantitative comparisons. These include traditional methods MDLatLRR [37] and TGFuse [38]; the GAN-based method GanMCC [33]; CNN-based methods TarDAL [7] and U2Fusion [39]; and Transformer-based architectures SwinFuse [40] and CDDFuse [25]. We also include DIVFusion [19] as a representative method designed for both IVIF and LVIE tasks. Furthermore, we incorporate several recently proposed frameworks: SPT [41], a SAM-based segmentation and fusion method; MLFuse [42], which employs a multi-scenario feature joint learning architecture; and LACT [43], an illumination-aware cross-modality fusion framework tailored for complex lighting environments.

Evaluation indicators: Six metrics were selected to evaluate image quality: Spatial Frequency (SF) [44] measures detail clarity through intensity variations; Average Gradient (AG) [45] assesses texture richness via gradient magnitude; Entropy (EN) [46] measures the information content of fused images; Standard Deviation (SD) [47] statistically evaluates contrast and pixel distribution; Visual Information Fidelity (VIF) [48] assesses perceptual fidelity relative to the human visual system; and Multi-Scale Structural Similarity Index (MS-SSIM) [49] captures structural information more comprehensively through multi-scale calculation.

Implementation Details: In this study, 1024 pairs of infrared and visible images were randomly selected from the MSRS dataset to train the proposed network. All experiments were implemented using the PyTorch (version 1.6.1) framework with the following configurations: the training process lasted for 200 epochs with a batch size of 4, and the learning rate was maintained at $1\times {10}^{-4}$ for both the generator and discriminators. The hardware platform utilized for all experiments were conducted on a workstation equipped with an Intel Core i5-14600KF CPU (Intel Corporation, Santa Clara, CA, USA), 32 GB of RAM, and an NVIDIA GeForce RTX 4060 GPU (NVIDIA Corporation, Santa Clara, CA, USA).

4.2. Comparison Experiment

To evaluate the effectiveness of the proposed method, we conducted comprehensive qualitative and quantitative comparisons with 11 SOTA methods on the MSRS dataset.

We selected one pair of normal-light and low-light images to demonstrate the fusion results of different algorithms in Figure 6.

Figure 6. Qualitative comparison of PSGAN with 11 SOTA methods on the 00012N scene and the 01502D scene from the MSRS dataset.

In the low-light scene of Figure 6, SwinFuse and U2Fusion exhibit poor performance, with the white vehicle being severely degraded in their fusion results. Among the remaining methods, only our approach, DIVFusion, SPT, and LACT manage to preserve the primary target structure with relative clarity. Specifically, LACT and SPT still suffer from insufficient overall brightness, failing to effectively reconstruct the background architectural structures. DIVFusion demonstrates noticeable dynamic range compression, which blurs the vehicle’s front and shadow regions while introducing significant noise. In contrast, our method produces images with moderate brightness, distinct vehicle contours, and sharp edges. Furthermore, the architectural structures and vegetation textures in the background are well-preserved.

In the normal-light scene of Figure 6, our method presents richer hierarchical details within the infrared salient target (highlighted by the blue box). Meanwhile, in the facial region marked by the red box, our method demonstrates superior edge sharpness and detail preservation. It significantly outperforms most competing methods, yielding a more natural and visually pleasing overall appearance.

To further evaluate the capability of different fusion methods in modeling and preserving frequency information, we applied the Fourier transform to the results of ten methods that exhibited superior visual performance in low-light scenarios. As illustrated in Figure 7, from the perspective of centered unwrapped phase images, SPT and MLFuse exhibit favorable frequency-domain responses characterized by smooth edge transitions and well-preserved high-frequency details; in contrast, the phase structures of MDLatLRR and TGFuse appear relatively blurred, lacking clear frequency transitions. Furthermore, GanMCC and LACT display severe phase discontinuities and chaotic texture patterns, revealing notable deficiencies in frequency-domain modeling. Conversely, CDDFuse and TarDAL maintain robust phase characteristics, including smooth natural transitions and clear structural representation. Notably, PSGAN demonstrates the most ideal unwrapped phase performance among all evaluated methods, featuring global continuity, a well-organized structure, and rich mid-to-high-frequency details with almost no abrupt transitions. These results highlight PSGAN’s comprehensive advantages in structural information preservation, texture enhancement, and frequency continuity modeling, reflecting its superior performance in the frequency-domain representation of structural and textural information.

Figure 7. Centered and unwrapped phase images obtained by Fourier transform of the pictures from Figure 6.

As shown in Figure 8, the results of 6 evaluation metrics on MSRS dataset indicate the relative performance of each method. Notably, PSGAN method achieves the best performance on EN, SF, SD, and AG metrics. EN and SD values indicate that PSGAN produces fused images with the highest information content and contrast, effectively addressing issues of brightness and detail distortion. Its outstanding performance on AG metric further confirms its advantage in preserving edge and texture information. Although PSGAN performs slightly lower than some SOTA methods on MS-SSIM metric, this may be attributed to its brightness enhancement strategy, which leads to differences in brightness and contrast distribution between fused images and the reference images. Nevertheless, the superior performance of PSGAN in information content (EN) and contrast (SD) demonstrates significant advantages in detail preservation and visual richness, thereby maintaining strong overall competitiveness.

Figure 8. Quantitative comparison results between our method and 11 SOTA algorithms on 361 image pairs from the MSRS dataset, visualized using a radar chart and a heatmap.

4.3. Generalization Performance

Generalization performance is a critical indicator of model robustness. To evaluate this, we conducted comparative experiments against 11 SOTA methods on three widely used benchmarks: LLVIP (a nighttime road dataset for low-light vision), M3FD (a multi-scene dataset), and TNO (a multispectral dataset for military scenarios). Under identical experimental settings as the MSRS training, we randomly selected 40, 50, and 30 image pairs from these datasets, respectively, for comprehensive testing.

4.3.1. Qualitative Evaluation

As shown in Figure 9, SwinFuse produces excessively blurred backgrounds, which may be attributed to the transformer architecture overly emphasizing global dark context. SPT enhances infrared targets and visible background components, yet it neglects the gradient-level information present in infrared images. U2Fusion, as a unified fusion framework, shows rather modest performance in IVIF tasks. Although DIVFusion provides good contrast and visual effects, it introduces substantial noise artifacts, especially in background regions. GanMCC, as a GAN-based method, exhibits high contrast but compromise the true similarity to the original image, resulting in blurred edge structures. TGFuse shows results similar to SPT, focusing excessively on infrared salient targets while ignoring background detail, resulting in information imbalance. MLFuse fails to fully integrate the detailed information from both infrared and visible images. Conversely, the proposed method excels in preserving salient targets and fine textures with high structural clarity. Notably, the flowerbed region in Figure 9 (blue box) demonstrates its superior restoration of delicate structures compared to other algorithms. This consistency across both daytime and nighttime scenarios further validates the model’s strong generalization and robustness across diverse environments.

Figure 9. Qualitative comparison of PSGAN with 11 SOTA methods on the M3FD, TNO, and LLVIP datasets.

4.3.2. Quantitative Evaluation

Table 1 summarizes the quantitative evaluation results of PSGAN on the LLVIP, M3FD, and TNO datasets. On LLVIP, PSGAN achieves competitive performance across most evaluation metrics, demonstrating its effectiveness in mitigating information loss under extreme low-light conditions. On M3FD, PSGAN ranks among the top three methods in four of the six evaluation metrics. On TNO, PSGAN achieves the highest scores in VIF and SD, while performing less favorably than some competing methods in terms of EN, SF, and AG. This result may reflect a trade-off between low-light enhancement and cross-scenario adaptability. PSGAN is primarily designed for severely degraded low-light environments, whereas TNO mainly consists of grayscale multispectral images that generally do not suffer from severe illumination degradation. Applying PSGAN to TNO may therefore result in a certain degree of over-enhancement. Excessive luminance adjustment may compress subtle mid-tone transitions and alter the original multi-scale gradient distribution, potentially leading to lower scores in metrics related to image information content and high-frequency details, such as EN, SF, and AG. Nevertheless, PSGAN’s high VIF and SD scores indicate its ability to preserve visually relevant information and produce high-contrast fused images on TNO.

Table 1. Quantitative evaluation on LLVIP, M3FD, and TNO datasets. Best, second-best, and third-best results are highlighted in red, blue, and green, respectively.

Method	EN	SF	SD	AG	VIF	MS-SSIM
LLVIP Dataset
TGFuse	6.957	0.0652	8.626	5.017	0.9032	0.9053
SwinFuse	7.248	0.0619	9.978	4.731	0.9425	0.9429
GanMCC	6.934	0.0570	9.553	6.303	0.6653	0.9045
MDLatLRR	7.083	0.0365	9.983	2.987	0.7417	0.8714
SPT	7.310	0.0635	9.523	4.827	0.9160	0.9343
TarDAL	7.338	0.1150	9.414	4.854	0.8418	0.8273
U2Fusion	6.742	0.0501	9.266	4.054	0.6952	0.9179
CDDFuse	7.356	0.0613	9.861	4.294	0.9285	0.9606
DIVFusion	6.540	0.0410	8.588	2.855	0.6963	0.8562
MLFuse	7.445	0.0749	10.030	4.318	0.8508	0.9217
LACT	7.037	0.0644	10.200	6.034	0.9060	0.9320
PSGAN (Ours)	7.414	0.1029	10.220	7.653	0.8877	0.9516
M3FD Dataset
TGFuse	6.764	0.0392	10.92	3.785	0.8552	0.8272
SwinFuse	6.836	0.0549	10.35	2.693	0.8307	0.8059
GanMCC	7.139	0.0675	10.38	3.310	0.8728	0.8448
MDLatLRR	6.574	0.0385	10.54	3.450	0.8552	0.7616
SPT	7.093	0.0517	10.34	2.523	0.8652	0.8840
TarDAL	7.376	0.0635	10.13	3.343	0.8263	0.9169
U2Fusion	6.990	0.0517	9.94	2.779	0.7767	0.9061
CDDFuse	7.367	0.0572	10.24	4.509	0.8580	0.7020
DIVFusion	7.375	0.0681	10.83	3.123	0.8483	0.8804
MLFuse	7.226	0.0489	9.727	3.386	0.9148	0.7109
LACT	7.049	0.0967	10.34	3.927	0.8301	0.7487
PSGAN (Ours)	7.364	0.0862	12.64	6.661	0.8834	0.8834
TNO Dataset
TGFuse	6.343	0.0357	9.448	2.175	1.0427	0.9231
SwinFuse	6.914	0.0389	8.833	2.997	0.7484	0.8149
GanMCC	6.780	0.0581	9.481	1.308	0.6406	0.7349
MDLatLRR	6.835	0.0520	9.500	2.626	0.9629	0.8403
SPT	6.983	0.0494	9.415	3.211	0.8561	0.8807
TarDAL	6.876	0.0408	9.289	2.491	0.7994	0.8858
U2Fusion	6.748	0.0396	9.309	1.892	0.6108	0.9286
CDDFuse	7.027	0.0344	9.636	2.829	0.6570	0.8996
DIVFusion	6.707	0.0457	9.117	2.246	0.6791	0.8818
MLFuse	7.143	0.0481	9.611	2.813	0.7772	0.8980
LACT	6.709	0.0431	9.124	2.202	0.9994	0.9150
PSGAN (Ours)	6.595	0.0482	10.200	2.485	1.0735	0.9098

4.3.3. DS-ANP Evaluation

To comparatively analyze the overall performance of different image fusion algorithms across multiple evaluation metrics, this paper develops a DS-based multi-criteria image fusion evaluation framework, in which the weight assignment is guided by the Analytic Network Process (ANP). The detailed process is as follows:

First, for each fusion method, a performance evaluation matrix is constructed using six selected metrics. Since these metrics are expressed in different units, min-max normalization is applied to map all values to the [0, 1] interval, ensuring comparability during subsequent fusion. Next, based on the hierarchical dependency concept of ANP, weights are assigned to each metric. In this study, the standard deviation of each metric across all methods is used as a measure of “discriminative power” to compute the weight vector. The weight vector is calculated as: $\omega =[{\omega }_{EN},{\omega }_{SF},{\omega }_{SD},{\omega }_{AG},{\omega }_{VIF},{\omega }_{MS-SSIM}]$. The metric weights across the three datasets are summarized in Table 2. Then, the basic probability assignment (BPA) function is constructed for each evaluation metric. Specifically, for a given normalized metric, the value corresponding to the i-th fusion method is regarded as the belief support for that method. After normalizing across all methods, a BPA vector $m\left(C\right)$ is obtained for that metric. To reflect the varying importance of different metrics, each BPA vector is weighted by its corresponding ANP-derived metric weight, resulting in a new weighted BPA representation.

Table 2. The weights and standard deviations of the six evaluation metrics on LLVIP, M3FD, and TNO datasets.

Metric	LLVIP		M3FD		TNO
	Weight	Std Dev	Weight	Std Dev	Weight	Std Dev
EN	0.1659	0.3154	0.1895	0.3327	0.1536	0.2647
SF	0.1527	0.2905	0.1709	0.3001	0.1721	0.2966
SD	0.1795	0.3414	0.1450	0.2547	0.1434	0.2471
AG	0.1481	0.2816	0.1521	0.2670	0.1587	0.2735
VIF	0.1946	0.3700	0.1410	0.2477	0.2062	0.3554
MS-SSIM	0.1592	0.3028	0.2015	0.3539	0.1660	0.2860

Then, basic probability assignment (BPA) vectors are constructed and weighted. DS combination rule is applied:

$$\mathit{m}\left(C\right)={\displaystyle \frac{1}{1-K}}\sum _{\begin{array}{c}A\cap B=C\end{array}}{\mathit{m}}_1\left(A\right)·{\mathit{m}}_2\left(B\right)$$

(13)

$$K=\sum _{\begin{array}{c}A\cap B=\varnothing \end{array}}{\mathit{m}}_1\left(A\right)·{\mathit{m}}_2\left(B\right)$$

(14)

Through the fusion of BPAs across six evaluation metrics, the overall belief score of each fusion method is obtained, serving as the basis for global ranking and performance comparison. As shown in the updated Table 3, PSGAN achieves the highest overall belief scores across the LLVIP ($1.86\times {10}^{-10}$), M3FD ($6.57\times {10}^{-10}$), and TNO ($3.44\times {10}^{-11}$) datasets, demonstrating its balanced multi-criteria superiority and strong generalization capability across diverse scenarios.

Table 3. Belief score comparison of different fusion methods on LLVIP, M3FD and TNO datasets.

Rank	LLVIP		M3FD		TNO
	Method	Score	Method	Score	Method	Score
1	PSGAN (Ours)	$1.86\times {10}^{-10}$	PSGAN (Ours)	$6.57\times {10}^{-10}$	PSGAN (Ours)	$3.44\times {10}^{-11}$
2	LACT	$2.65\times {10}^{-11}$	DIVFusion	$1.50\times {10}^{-11}$	SPT	$2.89\times {10}^{-11}$
3	SwinFuse	$2.22\times {10}^{-11}$	GanMCC	$8.62\times {10}^{-12}$	MLFuse	$2.65\times {10}^{-11}$
4	CDDFuse	$1.92\times {10}^{-11}$	TarDAL	$5.27\times {10}^{-12}$	MDLatLRR	$2.15\times {10}^{-11}$
5	MLFuse	$1.90\times {10}^{-11}$	LACT	$4.43\times {10}^{-12}$	LACT	$4.40\times {10}^{-12}$

4.4. Ablation Experiment

We conducted an ablation study on the MSRS dataset to validate the effectiveness of each key component in PSGAN. The quantitative results are presented in Table 4.

Table 4. Ablation evaluation on MSRS dataset. Best, second-best, and third-best results are highlighted in red, blue, and green, respectively.

Method	EN	SF	SD	AG	VIF	MS-SSIM
W/o P(f)	6.392	0.0485	9.245	4.104	0.6530	0.9005
W/o NAM	6.713	0.0513	9.809	4.509	0.7457	0.8967
W/o WTConv	6.858	0.0528	10.07	4.641	0.7156	0.9123
W/o $L_{vid}$	6.754	0.0449	8.962	3.712	0.6866	0.9178
Ours	6.967	0.0542	10.26	4.837	0.7310	0.9150

4.4.1. Pseudo-Sensing Module

We removed PSM, allowing the discriminator to directly perform adversarial training between ${\mathit{I}}_f$ and the ${\mathit{I}}_{ir}$ and ${\mathit{I}}_{vi}$, in order to evaluate the specific contribution of this module in the image fusion task. As shown in the quantitative results in Table 4, all metrics except MS-SSIM experience a significant decline, which strongly validates the critical role of PSM in enhancing global information representation and improving visual quality. In the absence of this module, the domain discrepancy between fused images and the source images increases significantly. As a result, the fusion output fails to effectively simulate the natural imaging process, leading to loss of image details and blurred textures, and ultimately causing a noticeable degradation in overall fusion quality.

4.4.2. Normalization-Based Attention Module and Wavelet Convolutions

To evaluate the specific contributions of the NAM and WTConv modules to the fusion performance, we removed these two modules individually and replaced them with standard convolutional layers to conduct ablation experiments. As shown in Table 4, after removing the NAM module, the EN, SF, SD, AG, and MS-SSIM metrics all decreased to varying degrees, indicating that this module plays a crucial role in enhancing image information representation, preserving edge details, and maintaining structural consistency. It is worth noting that the VIF metric increased from 0.7310 to 0.7457; however, this change does not imply an improvement in the overall quality of the fused image. Since VIF primarily measures the extent to which the fused image retains visual information from the source images, removing NAM weakens the network’s ability to selectively enhance and modulate features across different modalities. This may result in the retention of more local structural information similar to the source images, thereby leading to a slight increase in VIF. However, the decline in several other metrics indicates that NAM still plays a positive role in improving the overall quality of the fused image. Furthermore, after removing WTConv, the SF and AG metrics dropped significantly, while EN and VIF also decreased, further validating the effectiveness of this module in multiscale detail extraction, texture information preservation, and edge feature enhancement.

4.4.3. Illumination Discriminator

We removed the low-light enhancement module while retaining the dual-discriminator network architecture to evaluate the specific role of this module in the image fusion task. As shown in the quantitative results in Table 4, removing this module leads to a significant decline in SD, AG, and MS-SSIM metrics, along with noticeable decreases in SF and VIF, clearly demonstrating that the low-light enhancement module plays a critical role in enhancing image brightness, edge clarity, and overall visual quality.

4.5. Image Fusion for Object Detection

We employ YOLOv5s as the baseline object detector to evaluate the infrared and visible source images, as well as the fused images generated by 11 competing methods and PSGAN, aiming to assess the impact of different fusion methods on downstream object detection tasks. The quantitative results and visual comparisons of detection results are presented in Table 5 and Figure 10, respectively.

Table 5. Object detection performance on MSRS dataset.Best, second-best, and third-best results are highlighted in red, blue, and green, respectively.(mAP values refer to mAP@[0.5:0.95]).

Method	Precision	Recall	mAP-Person	mAP-Car	mAP-All
SwinFuse	0.926	0.839	0.569	0.707	0.638
GanMCC	0.826	0.841	0.354	0.634	0.444
CDDFuse	0.937	0.867	0.482	0.536	0.509
MDLatLRR	0.887	0.866	0.612	0.702	0.657
SPT	0.893	0.779	0.362	0.712	0.537
TarDAL	0.936	0.868	0.614	0.693	0.653
U2	0.935	0.895	0.624	0.735	0.679
DIVFusion	0.883	0.831	0.572	0.647	0.587
MLFuse	0.938	0.898	0.356	0.643	0.500
LACT	0.951	0.826	0.554	0.746	0.685
PSGAN (ours)	0.951	0.822	0.518	0.690	0.604

Figure 10. Visual comparison of object detection results between our method and 11 SOTA methods.

As shown in Table 5, different fusion methods exhibit varying trade-offs across detection metrics. PSGAN achieves the highest Precision, indicating its effectiveness in reducing false-positive detections. However, PSGAN obtains a Recall of 0.822 and an overall mAP@[0.5:0.95] of 0.604, which are lower than the mAP@[0.5:0.95] scores of LACT (0.685) and U2Fusion (0.679). In particular, its mAP for the “Person” category is only 0.518, suggesting certain limitations in pedestrian detection.

The discrepancy between Precision and mAP is particularly evident in pedestrian detection. This phenomenon may be associated with changes in the local contrast between targets and their surroundings during low-light enhancement. Specifically, pedestrian detection relies, to some extent, on the thermal radiation contrast between infrared targets and the surrounding background. Excessive enhancement of dark backgrounds may weaken the local saliency of pedestrians, thereby increasing the risk of missed detections. Furthermore, pedestrians are typically small in size and have relatively complex contours. Noise and local gradient variations introduced during dark-region enhancement may interfere with target feature extraction and bounding box localization, potentially reducing detection accuracy at higher IoU thresholds. In contrast, vehicles generally exhibit more distinct and regular geometric structures and can benefit from enhanced visible-light texture information. Consequently, PSGAN achieves an mAP of 0.690 for the “Car” category, demonstrating competitive performance.

The visual comparisons in Figure 10 further reveal that TarDAL and CDDFuse produce false-positive detections in certain scenarios. In contrast, PSGAN generally provides high detection confidence scores and well-aligned bounding boxes for successfully detected vehicles and pedestrians, which is consistent with its high Precision.

4.6. Efficiency Analysis

To evaluate the practical deployment potential of the proposed model in real-world scenarios, all efficiency evaluations are conducted on a hardware platform equipped with an NVIDIA GeForce RTX 4060 GPU (8 GB VRAM) (NVIDIA Corporation, Santa Clara, USA). As reported in Table 6, we compare the trainable parameters (Params), inference runtime per image pair, and floating-point operations (FLOPs) of PSGAN against competing SOTA methods.

Table 6. Computational efficiency comparison of different fusion methods.

Methods	Params (M)	Runtime (ms)	FLOPs (G)
SwinFuse	2.021	455.403	33.720
GanMCC	1.864	2.501	1160.541
SDNet	0.029	123.390	17.675
SPT	68.284	45.240	88.059
TarDAL	0.297	47.34	182.275
U2	0.659	1388.24	405.174
DIVFusion	4.408	7.506	3387.869
MLFuse	0.112	30.460	31.003
LACT	0.410	321.010	114.910
PSGAN (ours)	0.1274645	17.24	60.04

Crucially, although PSGAN incorporates a pseudo-sensing module (PSM), URetinex-Net, and three discriminators to enforce physical consistency and illumination enhancement during training, these auxiliary components are completely decoupled and discarded during inference. Consequently, only the lightweight fusion backbone (UNW-Fuser) is executed at test time. By leveraging parameter-free normalization-based attention (NAM) and efficient Haar wavelet convolutions (WTConv), PSGAN requires merely 0.127 M parameters, which is substantially more compact than Transformer-based models such as SwinFuse (2.021 M) and SAM-based SPT (68.284 M), as well as the low-light fusion network DIVFusion (4.408 M). In terms of computational complexity and speed, PSGAN achieves a competitive FLOP count of 60.04 G and an average inference time of 17.24 ms (over 58 FPS), running nearly $2.7\times$ faster than TarDAL (47.34 ms) and $18.6\times$ faster than LACT (321.010 ms). While GanMCC (2.501 ms) and DIVFusion (7.506 ms) exhibit shorter runtimes, they incur massive computational overheads (1160.541 G and 3387.869 G FLOPs, respectively) and suffer from inferior quantitative metrics and severe visual noise in low-light scenes. Overall, PSGAN achieves a highly favorable trade-off between computational efficiency and fusion quality, satisfying real-time processing requirements.

5. Conclusions

This paper proposes PSGAN, a generative adversarial network-based framework for infrared and low-light visible image fusion, aiming to alleviate the domain gap between different imaging modalities and improve fusion quality under challenging low-light conditions. Within this framework, both the fusion module (UNW-Fuser) and the pseudo-sensing module (PSM) adopt U-Net-like architectures to effectively extract multi-scale features while maintaining physical imaging consistency. Furthermore, by introducing a pseudo-sensing mechanism, the discriminators can directly measure the distribution discrepancies between source modality images and their corresponding pseudo-sensed images, thereby facilitating effective cross-modal information fusion.

Although PSGAN achieves promising fusion performance in nighttime and low-light scenarios, the experimental results also reveal certain limitations in cross-scenario generalization and downstream task adaptability. Specifically, since PSGAN is primarily optimized for low-light degraded environments, applying it to normally illuminated or minimally degraded scenes, such as those in the TNO dataset, may lead to a certain degree of over-enhancement, potentially altering the original structural gradient distribution. Moreover, in downstream object detection tasks, excessive enhancement of dark backgrounds may reduce the local thermal contrast between small infrared targets (e.g., pedestrians) and their surroundings, thereby affecting target distinguishability and potentially leading to a decrease in mean Average Precision (mAP).

To address these limitations, future work will focus on developing an illumination-aware adaptive gating mechanism that dynamically adjusts the enhancement strength according to the ambient illumination level of the input visible image. This approach is expected to improve the model’s adaptability to varying illumination conditions and further enhance the practical applicability and robustness of fused images in downstream object detection tasks.