A Lighting-Aware Infrared–Visible Image Fusion Network for Security Surveillance

Abstract

Infrared and visible image fusion combines the thermal cues captured by infrared sensors with the rich structural and texture information provided by visible images. This technique is particularly valuable for security surveillance, nighttime scene perception, and target recognition under challenging environmental conditions. Existing methods generally adopt fixed fusion strategies and neglect dynamic dual-modality changes under low illumination, overexposure and strong light interference, failing to preserve both thermal target saliency and visible structural details in fused images. To address this problem, this paper proposes a Lighting-Aware Spatial–Frequency Fusion Network (LASFNet) for security surveillance. It first estimates modality reliability across low-light, overexposed and infrared-salient regions, incorporating it into the fusion of frequency-domain amplitude and phase. Spatial infrared, visible and frequency-domain compensation features are then jointly fused to generate the output. Experiments on M3FD, MSRS and RoadScene datasets show that LASFNet achieves competitive fusion performance with strong cross-dataset generalization. On M3FD, YOLOv8s with LASFNet-fused inputs achieves 84.825% mAP@0.5 and 57.319% mAP@0.5:0.95, outperforming visible, infrared and YDTR-fused inputs. The proposed method balances target saliency and scene structure, providing more effective visual input for object detection under complex illumination.

1. Introduction

With the rapid development of intelligent security, intelligent transportation, and unmanned inspection, visual perception in complex environments has become an important research direction in computer vision. In practical security surveillance scenarios, cameras often need to operate for a long time under non-ideal conditions, such as nighttime low illumination, local strong light, backlighting, smoke occlusion, and complex weather. Under these conditions, a single-modal sensor usually has difficulty in stably acquiring complete and reliable target information. Visible images have high spatial resolution, but their imaging quality is easily affected by illumination conditions. In low-light or overexposed scenes, targets may often become difficult to recognize. In contrast, infrared images can highlight salient targets according to thermal radiation information and provide more stable detection performance at night and in weak-light environments. However, infrared images are insufficient in presenting the internal texture, background structure, and semantic information of targets, and relying only on infrared images is also inadequate for distinguishing target objects. Therefore, infrared and visible image fusion, which fully exploits the complementary advantages of the two modalities, is particularly important and can provide more reliable visual input for security perception and target recognition in complex environments [1,2]. As shown in Figure 1, the reliability of infrared and visible modalities varies under different local imaging conditions, motivating illumination-aware spatial–frequency fusion.

Figure 1. (a) Local imaging conditions (scene examples in the figure are taken from M3FD); (b) fusion mechanism of LASFNet; (c) relationship between multi-level evaluations.

In recent years, infrared and visible image fusion methods have made significant progress. Traditional methods usually rely on manually designed fusion rules, such as multi-scale transform, sparse representation, saliency detection, or optimization models [3,4,5,6]. These methods can achieve good visual results in some scenarios, but their performance relies on manually designed representations and fusion rules. Fixed fusion strategies struggle to adapt the relative contributions of infrared and visible information to local illumination variations, which may cause target loss or local artifacts under complex imaging conditions. With the development of deep learning, fusion methods based on convolutional neural networks, autoencoders, generative adversarial networks, and Transformers have gradually become mainstream [7,8,9,10,11,12,13]. These methods can learn cross-modal feature representations from data and have stronger nonlinear modeling ability than traditional methods. Some studies have further introduced frequency-domain analysis into fusion networks to enhance global structural representation and texture preservation [14,15]. However, most existing methods adopt relatively fixed fusion strategies, that is, they integrate infrared and visible information in a similar manner across different scenes, while paying limited attention to the influence of illumination changes on the reliability of the two modalities.

In security surveillance scenarios, the contributions of infrared and visible modalities are not fixed, but are closely related to local illumination conditions. In normally illuminated regions, visible images usually provide clearer textures and background details. In nighttime low-light regions, visible information may be severely degraded, and the infrared thermal response becomes more critical for maintaining target information. In locally overexposed regions or areas affected by strong light interference, the structural information in visible images may be damaged, while infrared images can still provide relatively stable target contours. If a fusion network ignores such dynamic changes in modality reliability, it may introduce unreliable visible textures in low-light regions or over-enhance infrared responses in background regions, leading to blurred details and less salient targets in the fused image. Therefore, infrared and visible image fusion methods for practical applications should not only focus on image-level visual quality, but also further verify their effectiveness in downstream detection tasks.

To address the above problems, this paper proposes an illumination-aware spatial–frequency fusion method for security surveillance scenarios. Different from fusion strategies that use fixed modality weights, the proposed method starts from local illumination reliability and models the influence of low-light regions, overexposed regions, and infrared-salient target regions on the fusion process. Specifically, this paper first designs an illumination-aware module to estimate the reliability weights of infrared and visible modalities in different regions according to the brightness degradation of visible images, the degree of local overexposure, and the salient response of infrared images. Then, a frequency-domain fusion process is introduced to adaptively adjust the amplitude and phase information, so as to enhance global structural representation and target response under complex illumination conditions. Finally, the fused image is generated by combining the spatial-domain features and the frequency-domain compensation features.

The main contributions of this paper are summarized as follows:

(1)

To address the problem of modality reliability variation caused by low illumination, overexposure, and complex lighting in security surveillance scenarios, an illumination-aware infrared and visible image fusion framework is proposed, enabling the fusion process to dynamically adjust infrared and visible information according to local illumination conditions.

(2)

A lighting-guided spatial–frequency fusion mechanism is designed to generate corresponding weights for different scenes and introduce them into the frequency-domain feature fusion process. Meanwhile, spatial-domain features are combined to reconstruct target details and background structures.

(3)

Systematic experiments are conducted on security-related infrared–visible datasets, including M3FD, MSRS, and RoadScene [16]. In addition to traditional fusion quality evaluation, downstream object detection experiments are further introduced to verify the practical value of the fused images for security perception tasks from the perspectives of mean Average Precision (mAP) and Precision.

2. Related Work

2.1. Traditional Image Fusion Methods

Early image fusion research mainly relies on manually designed representations and heuristic fusion rules. Representative approaches include multiscale transform (MST) methods [3,17,18], saliency-driven strategies [5], sparse representation (SR) models [4], subspace learning techniques, and optimization-based formulations.

Among them, MST-based methods such as Laplacian Pyramid (LP) [19] and Discrete Wavelet Transform (DWT) [17] perform fusion through multiscale decomposition, in which coefficients are combined according to predefined rules. Although these methods can preserve structural information to some extent, their performance is highly sensitive to the selected transform and fusion strategy. Saliency-based methods instead emphasize visually prominent responses to retain perceptually important regions, particularly thermal targets in infrared images.

SR- and subspace-based approaches, including PCA and NMF [2], provide compact representations of source images. However, these methods often involve iterative optimization and may suppress subtle but informative details. Optimization-based methods formulate fusion as an energy minimization problem with handcrafted constraints. This formulation offers greater flexibility but still depends heavily on prior assumptions.

Overall, traditional methods are generally interpretable and computationally manageable, but their reliance on manually designed priors limits their ability to model the complex nonlinear relationships between infrared and visible images, especially under challenging imaging conditions.

2.2. Deep Learning-Based Image Fusion

With the development of deep learning, image fusion has gradually shifted from handcrafted design to data-driven representation learning. Existing methods can be broadly divided into autoencoder (AE)-based methods [8,14,20,21], generative adversarial network (GAN)-based methods [9,22], Transformer-based methods [15,23,24,25], and unsupervised frameworks [10,26].

AE-based methods learn latent representations from multimodal inputs and reconstruct fused images through decoders. Although they are effective in extracting deep features, many still use relatively fixed operations at the fusion stage, which restricts their adaptability to changing scenes. GAN-based methods introduce adversarial learning to improve the visual quality of fused images. Nevertheless, their training process is often unstable, and balancing texture fidelity with structural preservation remains difficult.

More recently, Transformer-based models have attracted considerable attention because of their ability to capture long-range dependencies. SwinFusion [15], for example, employs hierarchical attention to balance global context and local detail preservation. Hybrid CNN–Transformer architectures such as YDTR [24] combine the local modeling capability of convolution with the global representation ability of attention mechanisms. Meanwhile, unsupervised frameworks such as U2Fusion [10] learn adaptive fusion representations without relying on labeled data.

Despite these advances [27,28,29,30], two issues remain insufficiently addressed. First, most methods still conduct feature extraction and interaction mainly in the spatial domain, leaving frequency-domain information underused. This limitation may weaken contrast and texture representation in the fused images. Second, many approaches implicitly assume that the relative contributions of infrared and visible images remain nearly unchanged across different scenes, which is often inconsistent with practical imaging conditions.

In real-world surveillance scenarios, visible images may be degraded by low illumination, overexposure, backlighting, or local glare, while the saliency of infrared targets also varies with environmental conditions. Motivated by these observations, we propose LASFNet, which estimates scene-dependent illumination characteristics and performs joint spatial–frequency fusion to adaptively balance the contributions of the two modalities.

3. Methodology

Infrared images highlight salient targets, such as pedestrians and vehicles, through differences in thermal radiation, but contain limited texture and background structure. Visible images provide richer edges, textures, and scene details, yet their information may degrade under low illumination, local overexposure, and strong-light interference. The contributions of the two modalities should therefore vary with local imaging conditions rather than remain fixed across the entire image. Based on this observation, we propose the Lighting-Aware Spatial–Frequency Fusion Network (LASFNet).

3.1. Overall Network Architecture

As shown in Figure 2, LASFNet mainly consists of a Lighting Condition Perception and Guidance module (LCPG), a Cross-Modal Spatial Fusion module (CMSF), an Illumination-Guided Frequency Fusion module (IGFF), and an Adaptive Residual Reconstruction Head (ARH). LASFNet first estimates the modality reliability under the current illumination condition. It then extracts complementary information in the spatial and frequency domains in parallel, and finally generates the fused image $I_f$ through the ARH.

Figure 2. Overall architecture of the proposed LASFNet.

3.2. Lighting Condition Perception and Guidance

The illumination-aware subnetwork replaces a single global illumination weight with spatially varying modality-weight maps. As illustrated in Figure 3, Let $I_{ir}$ and $I_{vi}$ denote the infrared image and the luminance channel of the visible image, respectively. The subnetwork produces two response maps for the infrared and visible modalities, which are normalized along the modality dimension:

$$\left[W_{ir},W_{vi}\right]={\mathrm{Softmax}}_m\left({\mathcal{N}}_{IA}\left(\mathcal{C}(I_{ir},I_{vi})\right)\right),W_{ir}+W_{vi}=1,$$

(1)

where $\mathcal{C}(·,·)$ denotes channel-wise concatenation, and ${\mathcal{N}}_{IA}$ represents the illumination-aware subnetwork. ${\mathrm{Softmax}}_m(·)$ performs normalization along the modality dimension, and $W_{ir}$ and $W_{vi}$ are the normalized contribution weights of the infrared and visible inputs. These weight maps provide continuous local guidance for modality selection throughout the fusion process.

Figure 3. Structure of the illumination-aware guidance module.

3.3. Cross-Modal Spatial Fusion

The CMSF branch performs progressive spatial fusion. Two modality-specific encoders first extract infrared and visible features, which are modulated by the corresponding local reliability weights:

$$\begin{array}{cc}\hfill F_{ir}& =E_{ir}(I_{ir})\odot (1+W_{ir}),\hfill \\ \hfill F_{vi}& =E_{vi}(I_{vi})\odot (1+W_{vi}),\hfill \end{array}$$

(2)

where $E_{ir}$ and $E_{vi}$ are the infrared and visible feature encoders, respectively. $F_{ir}$ and $F_{vi}$ denote the resulting deep features, and ⊙ represents element-wise multiplication.

To exchange complementary information without treating all cross-modal differences as equally useful, CMSF performs gated bidirectional compensation. At the i-th stage, the cross-modal difference and the corresponding compensated features are defined as

$$\begin{array}{cc}\hfill {\mathsf{\Delta }}^i& =F_{vi}^i-F_{ir}^i,g^i=\sigma \left(\mathrm{GAP}({\mathsf{\Delta }}^i)\right),\hfill \\ \hfill {\widehat{F}}_{ir}^i& =F_{ir}^i\oplus g^i\odot {\mathsf{\Delta }}^i,\hfill \\ \hfill {\widehat{F}}_{vi}^i& =F_{vi}^i\oplus g^i\odot \left(-{\mathsf{\Delta }}^i\right).\hfill \end{array}$$

(3)

where ${\widehat{F}}_{ir}^i$ and ${\widehat{F}}_{vi}^i$ are the compensated infrared and visible features at the i-th stage. GAP denotes global average pooling, $\sigma$ is the sigmoid function, and ⊕ denotes element-wise addition. The gate $g^i$ is broadcast over the spatial dimensions and controls complementary information exchange in opposite residual directions, thereby avoiding identical updates to the two modality features.

After progressive feature extraction, the compensated modality features are concatenated along the channel dimension and mapped to a spatial-domain representation:

$$\begin{array}{cc}\hfill F_f& =\mathcal{C}\left({\widehat{F}}_{ir},{\widehat{F}}_{vi}\right),\hfill \\ \hfill F_{spa}& ={\mathcal{R}}_{spa}(F_f),\hfill \end{array}$$

(4)

where $\mathcal{C}$ denotes channel-wise concatenation, $F_f$ is the concatenated feature, and ${\mathcal{R}}_{spa}$ is the spatial mapping block. The resulting $F_{spa}$ is subsequently integrated with the frequency-domain response.

3.4. Illumination-Guided Frequency Fusion

The IGFF branch models amplitude and phase information separately in the frequency domain. To account for condition-dependent changes in each modality, the local reliability weights are first applied to the source images:

$${\tilde{I}}_{ir}=I_{ir}\odot (1+W_{ir}),{\tilde{I}}_{vi}=I_{vi}\odot (1+W_{vi}).$$

(5)

A fast Fourier transform is then used to obtain the amplitude and phase spectra:

$$\begin{array}{cc}\hfill \left[A_{ir},A_{vi}\right]& =\left[\left|\mathrm{FFT}({\tilde{I}}_{ir})\right|,\left|\mathrm{FFT}({\tilde{I}}_{vi})\right|\right],\hfill \\ \hfill \left[P_{ir},P_{vi}\right]& =\left[\angle \mathrm{FFT}({\tilde{I}}_{ir}),\angle \mathrm{FFT}({\tilde{I}}_{vi})\right].\hfill \end{array}$$

(6)

where $A_{ir}$ and $A_{vi}$ are the amplitude spectra of the infrared and visible inputs, while $P_{ir}$ and $P_{vi}$ are their phase spectra. The operator $|·|$ returns the magnitude, and ∠ returns the element-wise phase angle.

The concatenated amplitude and phase features are fused by separate convolutional blocks. The resulting spectra are then transformed back to the spatial domain:

$$\begin{array}{cc}\hfill A_f& ={\mathrm{Conv}}_A\left(\mathcal{C}(A_{ir},A_{vi})\right),\hfill \\ \hfill P_f& ={\mathrm{Conv}}_P\left(\mathcal{C}(P_{ir},P_{vi})\right),\hfill \\ \hfill {\widehat{F}}_{fre}& =\mathrm{IFFT}(A_f,P_f),\hfill \end{array}$$

(7)

where ${\mathrm{Conv}}_A$ and ${\mathrm{Conv}}_P$ denote the convolutional fusion blocks for amplitude and phase, respectively. IFFT denotes the inverse fast Fourier transform, and ${\widehat{F}}_{fre}$ is the reconstructed frequency response.

Channel-wise maximum and mean operations are subsequently applied to the reconstructed response. Their outputs are concatenated and projected by a convolutional block:

$$F_{fre}=\mathrm{Conv}\left(\mathcal{C}\left({\max }_c({\widehat{F}}_{fre}),{\mathrm{mean}}_c({\widehat{F}}_{fre})\right)\right),$$

(8)

where ${\max }_c$ and ${\mathrm{mean}}_c$ are computed along the channel dimension. This projection produces a compact frequency-domain representation.

Finally, ARH integrates the spatial response produced by CMSF and the frequency response produced by IGFF:

$$I_f={\mathcal{R}}_{ARH}\left(\mathcal{C}(F_{spa},F_{fre})\right),$$

(9)

where $F_{spa}$ and $F_{fre}$ denote the spatial- and frequency-domain responses, respectively, and ${\mathcal{R}}_{ARH}$ is the reconstruction mapping of ARH.

3.5. Loss Function

To preserve infrared targets, visible details, and a reasonable brightness distribution in the fused image, the network is trained with three terms: content preservation loss, detail restoration loss, and illumination constraint loss. Let $I_{ir}$, $I_{vis}$, and $I_f$ denote the infrared image, the luminance component of the visible image, and the fused image, respectively. All images are normalized to $[0,1]$. The total fusion loss is defined as

$${\mathcal{L}}_{fusion}={\mathcal{L}}_{con}+{\mathcal{L}}_{det}+{\mathcal{L}}_{light}.$$

(10)

3.5.1. Content Preservation Loss

The content preservation loss constrains the overall intensity of the fused image. Let $A(I)$ denote a $15\times 15$ local mean filter applied to image I, and let $C(x)=\min (1,\max (0,x))$ clip the value to $[0,1]$. The local luminance of the visible image is given by

$${\overline{I}}_{vis}=A(I_{vis}).$$

The low-light mask $M_l$, overexposure mask $M_h$, and overall degradation mask $M_d$ are defined as

$$\begin{array}{cc}\hfill M_l& =C\left({\displaystyle \frac{{\tau }_l-{\overline{I}}_{vis}}{{\tau }_l+\epsilon }}\right),\hfill \\ \hfill M_h& =C\left({\displaystyle \frac{{\overline{I}}_{vis}-{\tau }_h}{1-{\tau }_h+\epsilon }}\right),\hfill \\ \hfill M_d& =C(M_l+M_h),\hfill \end{array}$$

(11)

where $\epsilon ={10}^{-6}$. The thresholds ${\tau }_l$ and ${\tau }_h$ are set to $0.2$ and $0.8$, respectively, for low-light and overexposed regions.

Using the infrared and visible contribution weights $W_{ir}$ and $W_{vis}$ obtained from Equation (1), a weighted image is constructed as

$$I_w=W_{ir}\odot I_{ir}+W_{vis}\odot I_{vis}.$$

The adaptive reference image is then defined by

$$I_{ref}=(1-M_d)\odot I_w+M_d\odot \max \left(I_{ir},(1-M_h)\odot I_{vis}\right).$$

(12)

In normally illuminated regions, $I_{ref}$ mainly follows the weighted combination of the two modalities. In low-light or overexposed regions, more information is retained from the infrared image and the stronger source response. The factor $(1-M_h)$ reduces the influence of overexposed visible pixels.

The content preservation loss is written as

$$\begin{array}{cc}\hfill {\mathcal{L}}_{con}=& {\lambda }_c{∥I_f-I_{ref}∥}_1\hfill \\ \hfill & +{\lambda }_s{∥W_{ir}\odot \left|I_f-I_{ir}\right|+W_{vis}\odot \left|I_f-I_{vis}\right|∥}_1.\hfill \end{array}$$

(13)

The first term keeps the fused image close to the adaptive reference, while the second encourages it to follow the relatively reliable modality in different regions.

3.5.2. Detail Restoration Loss

To preserve target boundaries and scene textures, gradient and frequency reconstruction constraints are introduced. Let $S_x$ and $S_y$ denote the horizontal and vertical Sobel operators. The gradient magnitude of image I is defined as

$$g(I)=\left|S_x\ast I\right|+\left|S_y\ast I\right|.$$

The reference gradient is

$$G=\max \left(g(I_{ir}),g(I_{vis})\right),$$

(14)

where ∗ denotes convolution and the maximum operation is performed pixel-wise. Since G selects the stronger gradient from the two source images, it contains both infrared target boundaries and visible texture information.

Let F denote the reconstructed response obtained by applying the inverse Fourier transform to the IGFF branch in Equation (7). $N(F)$ first takes the absolute value of F, averages it across channels, and then normalizes the result to $[0,1]$. We define

$$\begin{array}{cc}\hfill I_{freq}& =N(F),\hfill \\ \hfill I_{max}& =\max \left(I_{ir},I_{vis}\right).\hfill \end{array}$$

(15)

$I_{freq}$ and $I_{max}$ have the same spatial size and value range. The detail restoration loss is defined as

$${\mathcal{L}}_{det}={\lambda }_g{∥g(I_f)-G∥}_1+{\lambda }_f{∥I_{freq}-I_{max}∥}_1.$$

(16)

The first term recovers target boundaries and high-frequency textures, whereas the second constrains the frequency branch to retain the global structure of the source images.

3.5.3. Illumination Constraint Loss

The illumination constraint loss consists of target preservation, weight regularization, and exposure consistency terms.

First, a thermal target mask is constructed from the difference between the infrared image and its local mean:

$$S=N\left({\left[I_{ir}-A(I_{ir})\right]}_{+}\right),$$

where ${\left[x\right]}_{+}=\max (x,0)$ and $N(·)$ denotes normalization to $[0,1]$. The target preservation loss is defined as

$${\mathcal{L}}_{tar}={∥S\odot \left(I_f-I_{ir}\right)∥}_1.$$

(17)

This term preserves the thermal response of infrared-salient targets in the fused image. A total variation constraint is further applied to the modality weights:

$$TV(W)={∥{\partial }_{x}W∥}_1+{∥{\partial }_{y}W∥}_1.$$

The corresponding regularization term is

$${\mathcal{L}}_{reg}=TV(W_{ir})+TV(W_{vis}),$$

(18)

where ${\partial }_x$ and ${\partial }_y$ denote the first-order horizontal and vertical differences, respectively. This constraint suppresses abrupt changes in the modality weights between neighboring regions.

Finally, the exposure consistency loss is defined as

$$\begin{array}{cc}\hfill {\mathcal{L}}_{exp}=& {∥M_l\odot \left[A(I_f)-\max \left(A(I_{ir}),A(I_{vis})\right)\right]∥}_1\hfill \\ \hfill & +{∥M_h\odot \left[A(I_f)-A(I_{ir})\right]∥}_1.\hfill \end{array}$$

(19)

In low-light regions, the first term preserves the stronger local response from the two modalities. In overexposed visible regions, the second term makes the fused image rely more on the infrared image.

The illumination constraint loss is therefore given by

$${\mathcal{L}}_{light}={\lambda }_t{\mathcal{L}}_{tar}+{\lambda }_r{\mathcal{L}}_{reg}+{\lambda }_e{\mathcal{L}}_{exp}.$$

(20)

In our experiments, the loss weights are fixed as ${\lambda }_c=1.15$, ${\lambda }_s=0.9$, ${\lambda }_g=9.5$, ${\lambda }_f=0.8$, ${\lambda }_t=1.0$, ${\lambda }_r=0.55$, and ${\lambda }_e=1.25$. These coefficients balance intensity preservation, structure recovery, target enhancement, and illumination stability.

4. Experiments

4.1. Experimental Settings

4.1.1. Datasets and Splits

To evaluate the fusion performance of LASFNet and its generalization across datasets, experiments were conducted on three public infrared–visible image datasets: M3FD [16], MSRS [31], and RoadScene [10]. M3FD was used for training, validation, and testing, whereas MSRS and RoadScene were used only for external testing and were not involved in model training.

The M3FD dataset contains 4200 registered infrared–visible image pairs and provides annotations for six road-scene object categories, including pedestrians and vehicles. With the random seed fixed at 2026, the dataset was divided using image-level multilabel stratification into 3360 training pairs, 420 validation pairs, and 420 test pairs.

The MSRS dataset contains 1444 strictly registered infrared–visible image pairs collected from road, pedestrian, and building scenes under daytime, nighttime, and adverse weather conditions. It was used to examine the generalization of the fusion method in complex environments. The RoadScene dataset was constructed from FLIR video sequences and contains 221 registered image pairs, mainly covering typical driving scenes involving roads, vehicles, and pedestrians.

4.1.2. Data Preprocessing and Augmentation

During training, the infrared and visible images were synchronously processed to a size of $320\times 320$ pixels. The infrared image was converted to a single-channel grayscale image, while the visible image was transformed into the YCbCr color space. Since this study primarily focuses on illumination degradation, structural preservation, and the complementary relationship between infrared thermal responses and visible information, only the luminance channel Y was used as the visible input to the fusion network. This design allows the illumination-aware modules to directly model local brightness variations and structural information while avoiding additional interference from chrominance components. The original Cb and Cr channels were retained and combined with the fused luminance component to reconstruct the final color image. Therefore, the chromatic information of the visible image was preserved in the output, although it was not explicitly involved in the learned fusion process. We acknowledge that the individual RGB channels contain different spectral responses and may provide additional cues for certain objects. Explicit channel-wise spectral modeling is beyond the scope of the present work and will be further investigated in future studies. All input pixel values were normalized to $[0,1]$. Random cropping, horizontal flipping, and mild affine transformations were used during training to improve model robustness. The same geometric transformation parameters were applied to both modalities to preserve their spatial correspondence. No random data augmentation was used during validation or testing.

4.1.3. Implementation Details

The experiments were run on a Windows 11 workstation with an Intel Core i7-13650 CPU (Intel Corporation, Santa Clara, CA, USA) and an NVIDIA RTX 5060 GPU (NVIDIA Corporation, Santa Clara, CA, USA). The network was developed in Python 3.10.20 using the PyTorch 2.1.0 framework, and CUDA 12.6 was employed for GPU acceleration.Model parameters were optimized with Adam. The initial learning rate and batch size were set to $2\times {10}^{-4}$ and 16, respectively, and training continued for 50 epochs. To ensure reproducibility, the random seed was fixed at 2026 throughout the main experiments.

The loss function was organized into three groups: content preservation, illumination guidance, and structural regularization. The loss weights were empirically set to ${\lambda }_c=1.15$, ${\lambda }_s=0.9$, ${\lambda }_g=9.5$, ${\lambda }_f=0.8$, ${\lambda }_t=1.0$, ${\lambda }_r=0.55$, and ${\lambda }_e=1.25$.

4.1.4. Evaluation Metrics

Six commonly used no-reference image fusion metrics were adopted for quantitative evaluation: entropy ($\mathrm{EN}$) [32], standard deviation ($\mathrm{SD}$), spatial frequency ($\mathrm{SF}$) [33], mutual information ($\mathrm{MI}$) [34], visual information fidelity ($\mathrm{VIF}$) [35,36], and the edge-based fusion quality metric $Q^{AB/F}$ [37].

4.2. Comparison Experiments

LASFNet was compared with several representative methods, including DenseFuse [8], FusionGAN [9], IRFS [38], U2Fusion [10], SwinFusion [15], YDTR [24], and MFELF. Quantitative experiments were carried out on the M3FD, MSRS, and RoadScene datasets using six commonly adopted metrics, namely $\mathrm{EN}$, $\mathrm{SD}$, $\mathrm{SF}$, $\mathrm{MI}$, $\mathrm{VIF}$, and $Q^{AB/F}$. As these metrics characterize different aspects of the fused images, conclusions drawn from any single measure may be incomplete. The following analysis therefore considers both the overall performance ranking and the consistency of each method across the three datasets.

4.2.1. Results on the M3FD Dataset

Table 1 presents the quantitative comparison on the M3FD test set. LASFNet ranks first in $\mathrm{EN}$, $\mathrm{SF}$, $\mathrm{MI}$, and $\mathrm{VIF}$, with scores of 6.875, 15.571, 4.719, and 1.164, respectively. The gains in $\mathrm{SF}$ and $\mathrm{MI}$ suggest that the proposed method retains richer texture variations while transferring complementary information from the two source modalities. Its leading $\mathrm{VIF}$ score further indicates that more visually meaningful information is preserved in the fused result. Although the highest $\mathrm{SD}$ and $Q^{AB/F}$ scores are obtained by other methods, LASFNet remains competitive on both metrics and shows a more balanced overall performance.

Table 1. Quantitative results on the M3FD dataset.

Methods	$\mathbf{EN}$	$\mathbf{SD}$	$\mathbf{SF}$	$\mathbf{MI}$	$\mathbf{VIF}$	QAB/F
DenseFuse	6.410	31.670	9.370	2.880	0.537	0.410
FusionGAN	6.810	34.170	10.580	2.533	0.594	0.573
IRFS	6.738	31.505	11.270	3.719	0.618	0.515
U2Fusion	6.753	33.092	12.942	3.665	0.641	0.565
SwinFusion	6.823	40.284	10.543	4.241	0.948	0.673
YDTR	6.694	30.112	9.377	2.913	0.776	0.513
MFELF	6.751	33.034	10.676	2.694	0.953	0.791
LASFNet	6.875	35.513	15.571	4.719	1.164	0.694

Note: The best results are shown in bold, while the second-best results are underlined.

Figure 4 presents the qualitative comparison on the M3FD dataset.

Figure 4. Qualitative comparison on the M3FD dataset. (a) Infrared image; (b) visible image; (c) DenseFuse; (d) FusionGAN; (e) IRFS; (f) U2Fusion; (g) SwinFusion; (h) YDTR; (i) MFELF; and (j) LASFNet. The green boxes highlight infrared-salient targets for comparing target preservation, while the red boxes indicate local structural and texture regions for comparing detail preservation.

The visual results provide further evidence of this balance. Some competing methods strengthen infrared responses but suppress visible-light details in areas such as mountains and vegetation, whereas others preserve more background texture at the cost of weak contrast around thermal targets. In comparison, LASFNet makes pedestrians and other infrared-salient objects easier to distinguish without noticeably weakening scene structures or fine textures. The fused images also exhibit smoother brightness variation, with fewer abrupt transitions between highlighted targets and surrounding regions.

4.2.2. Results on the MSRS Dataset

To examine cross-dataset generalization, the model trained exclusively on M3FD was evaluated directly on MSRS without further fine-tuning. As reported in Table 2, LASFNet ranks first in $\mathrm{SD}$ and $\mathrm{SF}$ and second in $\mathrm{EN}$, $\mathrm{MI}$, $\mathrm{VIF}$, and $Q^{AB/F}$. This consistent performance indicates that the learned fusion strategy transfers well to previously unseen scenes, particularly in preserving contrast, texture variation, and visually informative content.

Table 2. Quantitative results on the MSRS dataset.

Methods	$\mathbf{EN}$	$\mathbf{SD}$	$\mathbf{SF}$	$\mathbf{MI}$	$\mathbf{VIF}$	QAB/F
DenseFuse	6.136	33.486	8.431	3.425	0.592	0.453
FusionGAN	5.839	28.434	8.049	3.107	0.581	0.254
IRFS	6.603	35.869	10.725	3.836	0.733	0.458
U2Fusion	6.456	35.223	8.608	3.437	0.659	0.451
SwinFusion	6.723	41.171	8.124	3.841	0.848	0.634
YDTR	6.627	35.367	9.407	4.372	0.976	0.746
MFELF	5.775	33.882	8.107	4.007	0.853	0.549
LASFNet	6.652	42.511	11.296	4.279	0.928	0.651

The nighttime examples further illustrate this advantage. Several competing methods generate dim fused images in which roads, buildings, and background regions near pedestrians are difficult to recognize, whereas others introduce excessive local enhancement and suppress details in bright areas. LASFNet preserves clear thermal responses of pedestrians while recovering road structures and environmental textures in poorly illuminated regions. It also alleviates local overexposure, making the fused results better suited to surveillance scenes containing both low-light areas and strong localized illumination.

Figure 5 presents the qualitative comparison on the MSRS dataset.

Figure 5. Qualitative comparison on the MSRS dataset. (a) Infrared image; (b) visible image; (c) DenseFuse; (d) FusionGAN; (e) IRFS; (f) U2Fusion; (g) SwinFusion; (h) YDTR; (i) MFELF; and (j) LASFNet. The green boxes highlight infrared-salient targets, such as pedestrians and bright thermal objects, while the red boxes indicate local structural and weak-texture regions for evaluating detail preservation.

4.2.3. Results on the RoadScene Dataset

Table 3 reports the quantitative results on the RoadScene dataset. LASFNet obtains the highest $\mathrm{SF}$ and $\mathrm{MI}$ scores, reaching 13.804 and 4.289, respectively. The strong performance on these two metrics indicates that the proposed method preserves rich spatial details while effectively combining complementary information from infrared and visible images.

Table 3. Quantitative results on the RoadScene dataset.

Methods	$\mathbf{EN}$	$\mathbf{SD}$	$\mathbf{SF}$	$\mathbf{MI}$	$\mathbf{VIF}$	QAB/F
DenseFuse	6.948	38.246	11.475	2.705	0.548	0.415
FusionGAN	6.473	31.434	9.430	2.837	0.362	0.313
IRFS	7.029	36.199	10.034	3.739	0.574	0.453
U2Fusion	7.039	39.146	13.214	3.306	0.613	0.521
SwinFusion	7.359	43.991	12.457	4.214	0.684	0.574
YDTR	6.914	35.428	10.569	4.154	0.554	0.452
MFELF	7.047	38.708	9.243	3.947	0.618	0.455
LASFNet	7.212	42.752	13.804	4.289	0.677	0.524

Although SwinFusion achieves slightly higher scores on several other metrics, LASFNet delivers a more balanced overall result. It is also worth noting that RoadScene is not included in the training set. The stable performance on this dataset therefore suggests that LASFNet can adapt reasonably well to variations in imaging conditions, scene distributions, and road environments.

Figure 6 presents the qualitative comparison on the RoadScene dataset.

Figure 6. Qualitative comparison on the RoadScene dataset. (a) Infrared image; (b) visible image; (c) DenseFuse; (d) FusionGAN; (e) IRFS; (f) U2Fusion; (g) SwinFusion; (h) YDTR; (i) MFELF; and (j) LASFNet. The green boxes highlight texture-rich regions, such as traffic signs, for evaluating visible-detail preservation, while the red boxes indicate distant targets and local structural regions for comparing target visibility and structural fidelity.

The visual comparison is consistent with the quantitative results. Some competing methods introduce brightness distortion around vehicles or weaken lane markings, object boundaries, and distant structures. LASFNet preserves clearer vehicle contours and road details while retaining salient thermal responses from the infrared image. Its fused results maintain moderate contrast without obvious local over-enhancement or texture smoothing, which agrees with the high $\mathrm{SF}$ and $\mathrm{MI}$ scores.

Table 4 compares the inference time, parameter count, and floating-point operations (FLOPs) of different fusion methods. LASFNet maintains a relatively low computational cost while achieving competitive fusion performance. This advantage mainly comes from the parameter-free FFT-based global modeling and lightweight frequency-domain operations, which avoid the substantial overhead introduced by complex attention-based architectures.

Table 4. Computational efficiency comparison.

Method	Time (s)	Params (M)	FLOPs (G)
FusionGAN	0.463	0.32	234.12
IRFS	0.002	0.24	73.74
U2Fusion	0.246	0.66	366.34
SwinFusion	2.734	0.56	148.94
YDTR	0.033	0.11	96.47
MFELF	0.070	0.37	114.06
LASFNet	0.012	0.16	67.34

As shown in Figure 7, LASFNet achieves a favorable balance between fusion performance and computational efficiency. It requires 67.34 G FLOPs, 0.16 M parameters, and an average inference time of 0.012 s per image pair. Among the compared methods, LASFNet has the lowest computational cost while maintaining competitive fusion performance.

Figure 7. Trade-off between fusion performance and computational complexity.

Compared with SwinFusion, LASFNet reduces the inference time from 2.734 s to 0.012 s and the computational cost from 148.94 G to 67.34 G FLOPs. Although IRFS is faster and YDTR has slightly fewer parameters, their fusion performance is lower. LASFNet provides a balanced trade-off among fusion quality, model size, and computational cost.

4.3. Ablation Study

Ablation experiments were conducted on M3FD to examine the contribution of the proposed spatial–frequency collaborative architecture. The image reconstruction module was kept unchanged in all ablation settings. Only LCPG, CMSF, and IGFF were removed in turn, while the remaining network structure and experimental settings were kept the same. The results are reported in Table 5.

Table 5. Ablation study results on the M3FD dataset.

Methods	$\mathbf{EN}$	$\mathbf{SD}$	$\mathbf{SF}$	$\mathbf{MI}$	$\mathbf{VIF}$	QAB/F
w/o LCPG	6.804	32.761	13.889	2.642	0.701	0.669
w/o CMSF	6.796	32.902	14.735	3.306	0.775	0.682
w/o IGFF	6.814	33.654	14.663	3.399	0.786	0.680
LASFNet	6.875	35.513	15.571	4.719	1.164	0.694

Figure 8 presents the qualitative comparison of the ablation study on the M3FD dataset.

Figure 8. Qualitative comparison of the ablation study on the M3FD dataset. (a,g) Infrared images; (b,h) visible images; (c,i) results without LCPG; (d,j) results without CMSF; (e,k) results without IGFF; and (f,l) LASFNet results. The red boxes indicate regions selected for local-detail comparison, while the green boxes mark the corresponding regions in the LASFNet results.

Removing any of the three modules led to decreases in the evaluation metrics, showing that each module contributes to fusion performance. The largest degradation occurred after removing LCPG: $\mathrm{MI}$ and $\mathrm{VIF}$ decreased from 4.719 and 1.164 to 2.642 and 0.701, respectively. This result shows that illumination priors are important for coordinating infrared and visible information and maintaining visual information fidelity. Without CMSF, $\mathrm{SF}$ decreased to 14.735, indicating that multiscale spatial feature fusion helps preserve object contours and local textures. Removing IGFF also reduced $\mathrm{MI}$, $\mathrm{VIF}$, and $\mathrm{SF}$, suggesting that frequency-domain interaction complements high-frequency details and cross-modal information that are not fully represented by the spatial branch. The complete LASFNet achieved the best results on all six metrics. LCPG, CMSF, and IGFF therefore contribute through illumination guidance, spatial structure modeling, and frequency-domain interaction, respectively, and jointly improve target saliency, structural preservation, and visual fidelity.

4.4. Downstream Object Detection

To further validate the effectiveness of the fused images for high-level vision tasks, downstream object detection experiments are conducted on the M3FD dataset. The inputs to the detector include visible images, infrared images, images fused by our LASFNet, and images fused by YDTR (which ranked second in the overall comparative evaluation). All experiments share the same data split and utilize the YOLOv8s detector. The input size is set to $640\times 640$, the batch size is 8, and the optimizer is AdamW, with the training process lasting for 150 epochs. To reduce the influence of random initialization and improve the reliability of the evaluation, all detection experiments are independently repeated using three random seeds 2024, 2025, and 2026, while keeping the training, validation, and test splits unchanged.The evaluation metrics used are mAP@0.5 and mAP@0.5:0.95.

As shown in Table 6, the mAP@0.5 and mAP@0.5:0.95 of LASFNet reach 84.825% and 57.319%, respectively. Compared to Visible, Infrared, and YDTR, these two metrics show improvements of 6.224/5.049, 6.406/3.562, and 1.964/1.229 percentage points, respectively. LASFNet achieves the highest AP in the Car, Bus, Motorcycle, and Truck categories. Notably, the AP for Motorcycle is 8.502 percentage points higher than that of YDTR. Although the performance in the People and Lamp categories is not the highest, the overall performance remains the best, indicating that the proposed method can more adequately preserve the complementary features required for detection.

Table 6. Detection performance and class-wise AP@0.5 (%) under different input formats.

Input Format	mAP@0.5	mAP@0.5:0.95	People	Car	Bus	Motorcycle	Lamp	Truck
Visible	78.601 ± 0.284	52.270 ± 0.391	71.198	92.578	90.101	71.183	65.094	81.449
Infrared	78.419 ± 0.317	53.757 ± 0.428	76.512	90.330	89.899	63.884	65.031	84.845
YDTR	82.861 ± 0.246	56.090 ± 0.365	82.221	90.397	90.204	72.681	71.175	90.488
LASFNet	84.825 ± 0.221	57.319 ± 0.337	80.459	92.704	92.162	81.183	69.992	92.449

The upper row of Figure 9 shows the results for scenario 03068. Both the Visible and YDTR inputs lead to false “People” detections on the road, whereas no such error appears in the Infrared or LASFNet results. Compared with the Infrared image, LASFNet retains more details in the road surface, trees, and roadside facilities, while preserving correct detections of cars and motorcycles.

Figure 9. Object detection comparison of four input formats in representative scenarios. The first row shows a scenario with detection differences, and the second row illustrates a scenario highlighting fusion quality.

The lower row corresponds to scenario 02760, which was captured in a low-light garage. In the Visible image, vehicles show limited contrast against the surrounding background. The Infrared image makes the thermal targets more prominent, but loses part of the structural information in ceiling pipes, pillars, and passage boundaries. LASFNet preserves both the vehicle contours and the surrounding scene structure, making the targets easier to distinguish without sacrificing environmental details. These results suggest that the information retained during fusion can also benefit downstream detection, providing more stable visual cues for safety-related perception under challenging conditions.

5. Conclusions

LASFNet differs from fusion methods with fixed modality contributions, as it can estimate spatially varying reliability weights according to local illumination conditions. The CMSF module designed in this paper captures complementary cross-modal information in the spatial domain to preserve target contours and scene details. Meanwhile, the IGFF module independently processes amplitude and phase information to improve frequency-domain representation capability under complex illumination. Finally, the adaptive reconstruction head integrates spatial and frequency features to generate the fused image. Extensive experiments conducted on the M3FD, MSRS and RoadScene datasets demonstrate that LASFNet achieves competitive quantitative metrics and visual performance and possesses favorable cross-dataset generalization ability. Ablation experiments further verify the effectiveness of the proposed modules. In addition, downstream object detection experiments prove that fused images can provide more effective visual information for target recognition in complex surveillance scenes.