Skip to Content
Remote SensingRemote Sensing
  • Article
  • Open Access

29 January 2026

22 Pages

Chrominance-Aware Multi-Resolution Network for Aerial Remote Sensing Image Fusion

,
,
and
School of Artificial Intelligence and School of Automation, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
*
Author to whom correspondence should be addressed.
This article belongs to the Section Remote Sensing Image Processing

Highlights

What are the main findings?
  • CMNet uses an enhanced Mamba-based multi-scale U-Net to jointly fuse infrared and visible intensity and visible chrominance; compared with intensity-only fusion, it improves texture detail while maintaining higher color fidelity.
  • The proposed Chrominance-Enhanced Fusion module produces more color-consistent fused images and achieves better results than existing methods on multiple quality metrics across public datasets.
What are the implications of the main findings?
  • The chrominance-aware fusion strategy reduces infrared-induced color distortion while preserving informative thermal cues, supporting clearer interpretation of fused remote-sensing imagery.
  • The proposed design provides a practical way to incorporate chrominance into infrared–visible fusion, which can be adapted to other multi-modal fusion settings where color stability is required.

Abstract

Spectral data obtained from upstream remote sensing tasks contain abundant complementary information. Infrared images are rich in radiative information, and visible images provide spatial details. Effective fusion of these two modalities improves the utilization of remote sensing data and provides a more comprehensive representation of target characteristics and texture details. The majority of current fusion methods focus primarily on intensity fusion between infrared and visible images. These methods ignore the chrominance information present in visible images and the interference introduced by infrared images on the color of fusion results. Consequently, the fused images exhibit inadequate color representation. To address these challenges, an infrared and visible image fusion method named Chrominance-Aware Multi-Resolution Network (CMNet) is proposed. CMNet integrates the Mamba module, which offers linear complexity and global awareness, into a U-Net framework to form the Multi-scale Spatial State Attention (MSSA) framework. Furthermore, the enhancement of the Mamba module through the design of the Chrominance-Enhanced Fusion (CEF) module leads to better color and detail representation in the fused image. Extensive experimental results show that the CMNet method delivers better performance compared to existing fusion methods across various evaluation metrics.

1. Introduction

Aerial remote sensing data exhibit significant diversity due to differences in sensor types, onboard platforms, and imaging modalities [1]. In remote sensing tasks, infrared images capture the thermal radiation features of targets, whereas visible images offer abundant color and structural details [2]. However, a single-modality image cannot simultaneously preserve texture, structural, and thermal information, which limits its ability to comprehensively represent surface features [3]. As shown in Figure 1, infrared and visible images provide complementary information, motivating their effective integration for more comprehensive scene representation. The use of fusion techniques to effectively integrate infrared and visible images enhances the complementarity of heterogeneous remote sensing data, reduces data redundancy, and improves the accuracy and completeness of Earth observation [4]. This method has emerged as an important research focus in the remote sensing field.
Figure 1. Given an infrared image I I R and a visible image I V I S , the fused image I F aims to preserve salient thermal targets from I I R while maintaining natural chrominance and structural details from I V I S , thereby retaining complementary information from both modalities.
Early multimodal image fusion methods primarily relied on predefined image decomposition and feature representation techniques, where source images were mapped into a feature space, fused according to specific strategies, and then reconstructed into a final image [5]. Based on different decomposition approaches, these methods can be categorized into multiscale transformation methods [6,7], sparse representation methods [8,9], subspace clustering methods [10], optimization-based methods [11,12], and hybrid methods [13]. However, these traditional methods have obvious limitations. Their fusion performance heavily depends on manually designed feature extraction techniques, which reduces their adaptability to complex application scenarios [5]. Moreover, the feature extraction process is limited in diversity, requires high computational resources, and lacks efficiency. These constraints reduce its effectiveness in achieving high-performance fusion [14]. In recent years, the rapid advancement of deep learning has significantly supported research on infrared and visible images fusion [15]. The main methods include autoencoder (AE)-based, convolutional neural network (CNN)-based, and generative adversarial network (GAN)-based approaches [16]. AE-based methods utilize an encoder–decoder structure to extract and rebuild image features while applying specific fusion strategies to optimize the results [17,18]. CNN-based methods usually build end-to-end networks and integrate specialized loss functions to improve fusion performance [19,20,21]. GAN-based methods utilize a generator to produce fused images and a discriminator to enhance image quality [22,23,24]. Additionally, certain methods incorporate image enhancement modules to recover details and contrast in visible images, improving their effectiveness in low-light conditions [21,25,26].
Although existing methods have made significant progress in fusion performance, limitations remain in color representation and global feature modeling. Mainstream approaches typically fuse the intensity channel of visible images with infrared images to emphasize thermal information [26]. This approach leads to unnatural color transitions during fusion by neglecting the imaging mechanism of the infrared image and the chromaticity information of the visible image. As shown in Figure 2, representative intensity-dominant fusion methods, such as Dif-Fusion [27], AGMFusion [24], and FAFusion [21], often produce local regions with noticeable color shift, over-saturation, or under/over-exposure compared with the visible input. These phenomena indicate that the fused results of such methods suffer from degraded color rendering and fail to sufficiently preserve the scene’s natural chrominance perception. The fusion result exhibits issues in color representation and does not adequately preserve the true color perception of the scene. Additionally, in multi-scale feature extraction and global information modeling, CNNs are widely used for their computational efficiency [28]. Nonetheless, their restricted receptive field limits their ability to capture global contextual information. In contrast, Transformers use self-attention mechanisms to provide robust global modeling capabilities [29]. However, their computational complexity increases quadratically with input resolution, which limits their efficiency in high-resolution tasks [30,31]. Recent research on State Space Models (SSMs), particularly the Mamba model [32], offers new insights for image fusion tasks. For aerial remote sensing image fusion, CNN-based architectures are computationally efficient but are constrained by local receptive fields, which limits their ability to model long-range spatial dependencies and maintain global scene consistency. Transformer-based methods alleviate this issue via self-attention, yet their computational and memory costs grow quadratically with image resolution, making them less practical for high-resolution aerial imagery. Mamba achieves global perception with linear complexity and demonstrates outstanding results and effectiveness in various computer vision tasks [33]. As an SSM-based architecture, Mamba models long-range interactions through selective state-space dynamics with linear complexity, enabling efficient global context aggregation while preserving scalability. These characteristics align well with aerial fusion requirements, where both large-scene global dependency modeling and efficient processing of high-resolution inputs are critical for retaining consistent structures, suppressing artifacts, and improving overall fusion quality.
Figure 2. Visual comparison illustrating chrominance-related failure cases in intensity-dominant infrared–visible image fusion. (a) IR and (b) VI are the source images. Representative methods, including (c) Dif-Fusion, (d) AGMFusion, and (e) HAIAFusion, exhibit noticeable chrominance imbalance in illuminated regions, which degrades the natural color perception of the scene. (f) The proposed method produces a more visually natural result while preserving thermal saliency and visible structural details.
Based on the previous analysis, this research introduces an aerial remote sensing image fusion method named Chrominance-Aware Multi-Resolution Network (CMNet), which is constructed using the state space model. The network includes two main parts: the Multi-scale Spatial State Attention (MSSA) framework and the Chrominance-Enhanced Fusion (CEF) module. The MSSA framework integrates a U-Net structure with the Mamba module to capture global features across multiple scales and ensure comprehensive global information extraction. The CEF module enhances chrominance representation in the main fusion process by integrating local features and introducing an auxiliary chrominance branch. This guarantees the fusion of both color completeness and intensity information in the final result. The primary contributions of this research are outlined as follows:
  • Based on the characteristics of the image fusion task, this study designs an interpretable network architecture that extends intensity-based fusion by introducing a dedicated chrominance branch. The network consists of three MSSA branches. Two branches progressively extract infrared and visible intensity features, while an additional chrominance branch is used to model chrominance features.
  • To effectively fuse intensity and chrominance information, this study extends the Mamba module to support multi-input scenarios and designs an innovative CEF module with chrominance feature fusion. The CEF module first fuses intensity features and then injects chrominance information to preserve chrominance consistency.
  • Comprehensive tests on three public datasets demonstrate that the proposed model outperforms existing techniques regarding fusion quality, color accuracy, and naturalness. The results indicate that CMNet confirms the validity of the proposed approach and attains competitive performance compared to other methods.
The rest of this article is structured as follows. Section 2 provides a brief review of related work. In Section 3, the detailed description of the proposed CMNet is presented. The results and experiments on three real remote sensing datasets appear in Section 4. Finally, the conclusion is provided in Section 5.

2. Related Work

2.1. Deep Learning-Based Image Fusion Methods

Existing methods for infrared and visible light image fusion typically convert visible light RGB images into luminance-chrominance color space and combine the luminance channel with infrared images to enhance modal consistency. Li et al. [34] propose the CASPFuse method, which improves modal consistency and structural preservation capabilities through cross-directional perceptual adversarial learning, though its performance remains limited under low-light conditions. To address this limitation, Park et al. [35] design the cross-modal Transformer model CMTFusion, which strengthens the retention of global complementary features, while its handling of local information remains insufficient. Consequently, to focus on local feature fusion, Tang et al. [30] propose DATFuse, which utilizes dual attention modules and unsupervised loss to avoid the complexity of rule design, though it performs poorly in enhancing low-light details. Zhang et al. [25] design EV-Fusion, which optimizes brightness and detail performance in low-light conditions through an unsupervised enhancement module and saliency map. Gao et al. [22] further propose DCDR-GAN, which significantly improves the information representation of fused images by integrating dense connections and adaptive normalization strategies. Additionally, Wu et al. [23] enhance image clarity in complex environments by designing DCFusion based on frequency-domain decomposition, while Hu et al. [20] propose PFCFuse, which integrates Poolformer and convolutional networks to reduce model parameters and improve fusion accuracy. To enhance edge expressiveness, Chang et al. [36] propose L ⁠ 2 Net, which improves edge feature representation through large-kernel feature extraction and edge enhancement modules. Furthermore, Li et al. [31] design RITFusion, which combines modal interaction and morphological loss to significantly improve the retention of thermal targets under adverse conditions. However, these methods primarily focus on intensity information during fusion design, with limited utilization of chrominance channel information. This limitation reduces the comprehensiveness of fusion results.

2.2. State Space Model

State space models (SSM) [37] originate from traditional control theory and serve as fundamental methods for dynamic system modeling. Recently, they show significant value in the field of deep learning. SSM uses linear ordinary differential equations (ODE) to describe the dynamic relationships among inputs, hidden states, and outputs. This formulation provides linear scalability in modeling long-range dependencies. Structured state space sequence models, represented by S4 [38], introduce HiPPO techniques to significantly reduce computational complexity and narrow the performance gap with Transformers. Subsequent models, such as S5 [39] and H3 [40], further enhance their applicability in language processing and time series tasks. Mamba [32], as a key advancement in SSM, incorporates dynamic feature extraction and data-dependent mechanisms to substantially improve global perception capabilities and computational efficiency. It outperforms Transformers in natural language processing and broadens its applications to areas like image classification, video analysis, and medical image segmentation. To address the limitation of insufficient global perception in 2D visual tasks, Vision Mamba [41] proposes a bidirectional flattening strategy to achieve more comprehensive global modeling. VMamba [42] enhances spatial connections through four-directional modeling, significantly improving performance in visual tasks. In remote sensing, models such as RSCaMam [43], RSMamba [44], and Pan-Mamba [45] have been effectively applied to change detection, classification, and pansharpening tasks. These models demonstrate the potential of SSM in multimodal information fusion.

2.3. Color-Fidelity Oriented Image Fusion Methods

Recent infrared and visible light image fusion research has increasingly emphasized color fidelity and chrominance preservation, since intensity-dominant fusion often introduces color shift, over-saturation, or exposure imbalance in RGB-visible scenes. Dif-Fusion explicitly targets high color fidelity and introduces a dedicated evaluation perspective to quantify color-related quality in infrared and visible light image fusion results [27]. Beyond metric-oriented designs, several methods enforce chrominance consistency through explicit color modeling strategies. For example, PHFuse focuses on preserving hue information while integrating complementary thermal and structural cues, which is particularly beneficial in low-light environments [46]. In addition, FCDFusion further addresses chromatic distortion by explicitly constraining color deviation during fusion, aiming to suppress infrared-induced color shift and maintain perceptual color consistency in the fused results [47]. Related color-aware nighttime fusion frameworks incorporate explicit color constraints, such as color-consistency losses or illumination-aware mechanisms, to prevent chrominance degradation and improve visual realism under challenging lighting conditions [48,49]. These studies collectively motivate chrominance-aware designs that decouple intensity and structural fusion from chrominance preservation for visually stable and color-faithful infrared and visible light image fusion.

3. Method

3.1. Framework Overview

This study proposes CMNet, a fusion network that fully utilizes chrominance information to achieve effective fusion of infrared and visible images in low-light environments. The network is designed to preserve fine texture details while enhancing and balancing color information. It consists of two key components: the MSSA framework and the CEF module. Figure 3 presents the overall workflow of the network. The core idea of CMNet is to enhance the texture details and visual color fidelity in the fused image. It takes infrared and visible images as input, decomposes the visible image into intensity and chrominance components, and then feeds the intensity component, the infrared image, and the chrominance component into the different MSSA frameworks, respectively. The MSSA framework employs a multi-scale feature extraction mechanism based on the U-Net architecture to effectively integrate multimodal features. It extracts feature maps at multiple scales and integrates them using the Mamba attention mechanism. Skip connections between scales preserve the integrity and consistency of information transfer. More specifically, the infrared and intensity branches are devoted to extracting structural textures and thermal saliency by jointly modeling the visible luminance component and the infrared modality. In contrast, the chrominance branch is designed to learn a chrominance-preserving representation from the CrCb channel, which encode color-difference cues that are relatively decoupled from illumination intensity. The chrominance features are not intended to compete with the intensity channels for structural content; instead, they serve as complementary chromatic context that constrains the reconstruction toward visually natural color appearance and suppresses infrared-induced color shift. Consequently, the overall objective establishes a clear division of roles: the intensity and infrared branches concentrates on structure and saliency, whereas the chrominance pathway regularizes the color distribution, and their combination yields fused results with both faithful details and consistent chrominance. After multi-scale feature extraction, the extracted features are fed into the CEF module. In this module, the intensity channel is first fused with the infrared channel. The resulting intensity information is then combined with the chrominance data to generate the final fused image.
Figure 3. The overall workflow of the proposed method is shown in the figure. First, all bands undergo feature extraction and reconstruction through the MSSA framework. The MSSA framework integrates a U-Net architecture for multi-scale feature extraction, incorporating the 4-Directional Mamba Block after each U-Net layer to comprehensively capture multi-scale global features. Next, the CEF module is applied to fuse the features of chrominance and intensity information. Finally, the fused image is obtained.

3.2. Multi-Scale Spatial State Attention Framework

The input visible light image is denoted as I VIS ∈ R H × W × C , where H, W, and C represent height, width, and the number of channels, respectively, with C = 3 in this case. After decomposition, the original RGB three channels are converted into I CrCb ∈ R H × W × 2 , which reflects chrominance information, and I Y ∈ R H × W × 1 , which reflects intensity information. Among these, I Y and the single-channel infrared image I IR ∈ R H × W × 1 , which also reflect intensity information, exhibit high similarity in terms of information representation. Therefore, applying the same feature extraction process to both maintains consistency in feature distribution within the network and helps prevent feature misalignment during the fusion process. As illustrated in the intensity and chrominance branches of Figure 1, the Multi-scale Spatial State Attention (MSSA) framework adopts the U-Net network structure for multi-scale feature extraction to enhance the diversity and completeness of extracted features. In each layer of the U-Net network structure, the design incorporates sampling layers and 4-Directional Mamba Block layers, as illustrated in Figure 4. The following sections present a detailed explanation of the specific processing steps.
Figure 4. The 4-Directional Mamba Block are illustrated in the figure. It models dependencies along four spatial directions: left-to-right, right-to-left, top-to-bottom, and bottom-to-top.
(1) 4-Directional Mamba Block: The key component of the Mamba module is its structured SSM combined with a selective input mechanism for dynamic sequence modeling. The feature map F in ∈ R H × W × C obtained from the preceding U-Net layer is first normalized using layer normalization in the 4-Directional Mamba Block to standardize the feature distribution. Two parallel 1 × 1 convolution layers then generate two feature maps, denoted as X ∈ R H × W × C and Z ∈ R H × W × C . The channel dimension of the 1 × 1 convolutions is determined by the location of the 4-Directional Mamba Block within the U-Net architecture and is consistent with the corresponding U-Net layer. Subsequently, X is flattened along four spatial directions, resulting in four feature sequences: x 1 , x 2 , x 3 , and x 4 . Each flattened feature matrix has H · W positions, where each position corresponds to a C-dimensional feature vector, yielding a representation of dimension H W × C . These sequences pass through four independent SSM modules and generate outputs y 1 , y 2 , y 3 , y 4 . This process is expressed as
x i = Flatten i ( X ) y i = SSM i ( x i ) , i = 1 , 2 , 3 , 4 ,
where Flatten i ( · ) denotes the flattening operation along the i-th direction, and SSM i ( · ) represents the processing result of the i-th directional SSM module. The outputs of the four SSM modules are then unflattened and combined to generate a new feature map Y ∈ R H × W × C . To incorporate the selective modeling mechanism inherent to the Mamba architecture, the feature map Z is used as a dynamic gating signal. Specifically, the output feature map Y , produced by the directional state-space modeling, is modulated via element-wise multiplication with the sigmoid-activated Z , thereby enabling selective information flow. The gated result is then passed through a 1 × 1 convolution layer Conv 1 × 1 ( · ) , and the output is added to the original input F in to obtain the final output feature map F out ∈ R H × W × C . This process is expressed as
F out = Conv 1 × 1 ( Y ⊙ σ ( Z ) ) + F in ,
where ⊙ represents the element-wise multiplication operation, and σ ( · ) denotes the Sigmoid activation function.
(2) Sampling Layer: The downsampling process applies 2 × 2 max pooling to retain salient features while reducing feature map size, which improves computational efficiency. A 1 × 1 convolution follows to adjust the number of feature channels, reduce parameter complexity, and enhance both computational efficiency and feature representation. A subsequent 3 × 3 convolution extracts features, captures local details, and enriches feature representation. The entire process employs LeakyReLU as the activation function to mitigate gradient vanishing issues and enhance the nonlinear representation of features. The 4-Directional Mamba Block processes the resulting feature maps and enables global attention perception. This mechanism captures multi-scale contextual information and enhances global feature awareness. The downsampling process in the MSSA framework is formulated as
F D n + 1 = Mamba Conv 3 × 3 Conv 1 × 1 Maxpool F D n ,
where F D n denotes the downsampled feature map at the n-th layer, F D n + 1 denotes the input features for the next layer, and Mamba ( · ) , Conv ( · ) , and Maxpool ( · ) represent the 4-Directional Mamba Block, convolution operation, and max pooling operation, respectively. Correspondingly, the upsampling process reconstructs feature maps using bicubic interpolation ( Bicubic ( · ) ), which generates smoother and higher-quality feature maps. The other structures in the upsampling process remain consistent with those in the downsampling process. These structures include 1 × 1 and 3 × 3 convolution operations as well as the 4-Directional Mamba Block. Additionally, skip connections ( F U n + F D n ) are introduced during upsampling to ensure the completeness of feature transmission and the continuity of semantic information while enhancing the feature reuse capability of the model. The upsampling process in the MSSA framework is represented as
F U n + 1 = Mamba Conv 3 × 3 Conv 1 × 1 Bicubic F U n + F D n ,
where F U n denotes the upsampled feature map at the n-th layer, F U n + 1 denotes the input features for the next layer, and Bicubic ( · ) represent the bicubic interpolation operation.

3.3. Chrominance-Enhanced Fusion Module

To simultaneously retain the structural information of infrared images and the brightness and color information of visible images in multimodal image fusion tasks, the CEF module is designed. This module consists of two parts: intensity information fusion part and chrominance information fusion part. Intensity fusion processes the features from the intensity branch of the MSSA framework, while chrominance fusion further integrates the features from the chrominance branch.
The overall workflow is shown in Figure 5. The design enables effective collaboration between intensity and color information. The inputs include the intensity channel of infrared image F in I IR , the intensity channel of the visible light image F in I Y ∈ R H × W × 1 , and the chrominance channel F in I CrCb ∈ R H × W × 2 . The initial feature maps of these channels are processed using layer normalization and convolution operations to generate
X m = Conv x ( Norm ( F m in ) ) , m ∈ { I Y , I CrCb , I IR } Z n = Conv z ( Norm ( F n in ) ) , n ∈ { I Y , I IR } ,
where Conv x ( · ) and Conv z ( · ) denote two independent convolution operations for the input feature maps, and Norm ( · ) represents layer normalization. After normalization and convolution, the resulting feature maps are denoted as X m . Subsequently, the feature maps X m are flattened along four predefined directions to produce one-dimensional sequences x m i with i ∈ { 1 , 2 , 3 , 4 } . This directional flattening converts each 2D feature map into a set of directional token sequences, enabling long-range modeling in the subsequent state-space module. Through cross-modal interaction and feature fusion, the x I Y i and x I IR i are fused, and the outputs y I a i and y I b i are generated. This process is expressed as
y I a i = DSSM i ( x I Y i , x I IR i ) y I b i = DSSM i ( x I IR i , x I Y i ) , i = 1 , 2 , 3 , 4 ,
where DSSM i ( · ) denotes the Dual-input State Space Model (DSSM) in the i-th direction, which is extended to accommodate dual inputs and performs feature interaction and fusion between the infrared and visible sequences. Specifically, DSSM is instantiated by extending the SSM block in Figure 4 from a single-stream input to a dual-stream input ( x , z ) , where x is treated as the primary token sequence for state propagation and z is introduced as an auxiliary embedding sequence for conditioning. Specifically, the auxiliary stream is equipped with an additional projection branch and is injected into the parameterization of the selective state-space scan so that the input-dependent dynamics used to scan x are modulated by z , thereby enabling cross-stream feature interaction and long-range dependency modeling. Importantly, this extension does not alter the intrinsic state transition form of the SSM; the same state-space transition mechanism is retained, while only the input interface and the input-dependent parameter generation are augmented to support dual-input conditioning. Subsequently, the chrominance channel features X I CrCb are combined with the fused intensity features and input into the DSSM module to supplement the chrominance information of the fused image I. The outputs of the DSSM modules are reconstructed into feature maps through the unflattening operation and are directly summed across the four directions to form the aggregated feature representation. The fusion outputs are
F out a = Conv 1 × 1 ( Y a · SiLU ( Z I Y ) ) + F in I Y F out b = Conv 1 × 1 ( Y b · SiLU ( Z I IR ) ) + F in I IR ,
where SiLU stands for the SiLU activation function. The final fused feature map, denoted as F out I F , is obtained by applying a 1 × 1 convolution to the sum of F out a and F out b . This module combines intensity and chrominance information. It effectively preserves the structural information of infrared images, the brightness information of visible images, and their rich color content. The fused result presents complete and precise visual expressiveness for multimodal image fusion.
Figure 5. The proposed CEF module is illustrated in the figure. Built upon the 4-Directional Mamba Block, the module jointly processes the luminance, infrared, and chrominance features F in I Y , F in I IR , and F in I CrCb extracted by the MSSA framework through cross-stream long-range interaction modeling. It incorporates chrominance channel features and generates the final fused result. This process enhances the color expressiveness and preserves more image details.

3.4. Loss Functions

(1) Total Loss: This study designs multiple loss functions to guide effective feature extraction during training. These functions include edge loss L edge , structural loss L SSIM , pixel loss L pixel , and color balance loss L color . The total loss function L total combines these sub-losses with weighted coefficients to optimize the overall performance of multimodal image fusion. It is defined as
L total = α 1 L edge + α 2 L SSIM + α 3 L p i x e l + α 4 L color ,
where α 1 , α 2 , α 3 , α 4 are the weights of the sub-losses. Each weight controls the contribution of its corresponding loss term to the total loss. In this study, the optimal values are set as α 1 = 10.0 , α 2 = 3.0 , α 3 = 1.0 , and α 4 = 5.0 , which are determined according to the functional roles and relative numerical scales of the individual loss terms to ensure balanced gradient contributions during training.
(2) Edge Loss: The edge loss preserves and enhances edge information in images to improve the detail representation of fused images. Sobel operators extract edges in the horizontal and vertical directions, and the edge loss is calculated as
L edge = 1 H × W ∥ ∇ ( I F ) − max ( ∇ I IR , ∇ I VIS ) ∥ 1 ,
where I F is the fused image, ∇ denotes the Sobel operator, and max ( · ) represents the element-wise maximum operation.
(3) Structural Loss: Structural loss measures the structural similarity between the fused image and the source images to ensure consistency in structural information. It is defined as:
L SSIM = 1 2 ( 1 − SSIM ( I IR , I F ) ) + 1 2 ( 1 − SSIM ( I VIS , I F ) ) ,
where SSIM ( A , B ) computes the structural similarity index between images A and B using a Gaussian window. In implementation, the SSIM window size is set to 11 with the reduction mode mean. The Gaussian kernel uses σ = 1.5 , the stability constants are set to C 1 = ( 0.01 · L ) 2 and C 2 = ( 0.03 · L ) 2 , and ϵ = 10 − 12 is adopted for numerical stability, where L denotes the dynamic range. Since the inputs are normalized, L is set to 1.0 . SSIM follows the standard decomposition into luminance l ( A , B ) α , contrast c ( A , B ) β , and structure s ( A , B ) γ components:
SSIM ( A , B ) = l ( A , B ) α c ( A , B ) β s ( A , B ) γ ,
where the exponents are set to α = β = γ = 1 , assigning equal importance to luminance, contrast, and structural similarity. The SSIM score is computed independently per channel and then averaged with equal weights via the mean reduction.
(4) Pixel Loss: Pixel loss measures the pixel-level intensity differences between the fused image and the source images using the L 1 -norm. This loss contributes to accurate preservation of brightness and contrast. It is expressed as
L p i x e l = max ( ∥ I F − I IR ∥ 1 , ∥ I F − I VIS ∥ 1 ) ,
where ∥ · ∥ 1 denotes the L 1 -norm. This norm represents the sum of the absolute differences between corresponding pixel values.
(5) Color Balance Loss: Color balance loss, based on the Gray World theory, minimizes the mean differences among the RGB channels of the fused image to achieve color balance. It is defined as
L color = ∥ J R − J G ∥ 2 2 + ∥ J R − J B ∥ 2 2 + ∥ J G − J B ∥ 2 2 ,
where J R , J G , J B represent the mean values of the red, green, and blue channels of the fused image, respectively.
J X = 1 H · W ∑ i = 1 H ∑ j = 1 W X i , j , X = R , G , B ,
where R i , j , G i , j , B i , j denote the pixel values in the red, green, and blue channels of the fused image, respectively.

4. Experiment

4.1. Datasets

(1) DroneVehicle Dataset: The DroneVehicle [50] dataset is a multimodal dataset collected by unmanned aerial vehicles (UAVs). It consists of both RGB and infrared (IR) images used for vehicle detection. The dataset includes 56,878 images, equally distributed between RGB and infrared modalities. It offers detailed annotations for five vehicle types: truck, car, van, bus, and freight car, with a total of 389,779 labeled objects in the RGB domain and 428,086 in the infrared domain.
(2) RGBT-Tiny Dataset: The RGBT-Tiny [51] dataset is a large-scale visible and infrared benchmark for small object detection. It consists of 115 pairs of high-quality image sequences, with a total of 93,000 frames. It features approximately 1.2 million manually labeled objects spanning 7 target categories across 8 different scenarios. More than 81% of the targets have sizes smaller than 16 × 16 pixels. This characteristic makes the dataset particularly suitable for small object detection and multimodal image fusion research. Each target is annotated with bounding boxes and tracking IDs. These annotations offer a valuable resource for studying RGB-T fusion, object detection, and object tracking under complex environments.
(3) VTUAV Dataset: The VTUAV [52] (Visible-Thermal UAV Tracking) dataset is a large-scale visible-infrared dataset created specifically for UAV-based target tracking. It includes 500 sequences with 1.7 million high-resolution (1920 × 1080) image pairs, collected under various conditions. This dataset offers a wide range of scenarios for assessing RGB-T tracking algorithms and serves as an essential benchmark for object tracking and fusion-based remote sensing applications.
During network training, the training set of the DroneVehicle dataset is used for model training. Given that the DroneVehicle, RGBT-Tiny, and VTUAV datasets are relatively large, an equidistant sampling strategy is applied to randomly select 100 samples from each dataset as the test set to ensure fairness and representativeness in the testing phase. Note that the images in the DroneVehicle dataset contain a 100-pixel-wide white border, which is cropped during both training and testing to prevent invalid regions from affecting the experimental results. Additionally, the image resolution of the VTUAV dataset is 1920 × 1080, whereas the resolutions of the DroneVehicle and RGBT-Tiny datasets are 640 × 512. To maintain consistency across datasets, the sampled test images from VTUAV are resized to match the dimensions of the other datasets using bilinear interpolation for downsampling.

4.2. Implementation Details and Metrics

To train the network, the dataset is prepared, the experimental environment is configured, and relevant hyperparameters are adjusted. The training is conducted using Python 3.10 and the PyTorch 2.0 framework on an NVIDIA 4090D GPU (NVIDIA Corporation, Santa Clara, CA, USA). The network is trained on 17,990 paired infrared–visible training images from the predefined split of the DroneVehicle dataset, and all images are normalized to improve generalization. During training, paired patches of size 128 × 128 are extracted via random cropping from aligned image pairs, and no additional data augmentation is applied. The network is optimized using Adam optimizer implemented in the PyTorch 2.0 framework. The model is trained for 500 epochs with a batch size of 10 and an initial learning rate of 5 × 10 − 4 , which decays by a factor of 0.8 every 50 epochs.
To comprehensively evaluate the quality of fused images, the following metrics are utilized. Visual Information Fidelity (VIF) measures the structural and perceptual similarity between the fused and source images. Mean Squared Error (MSE) quantifies the pixel-wise reconstruction error, with lower values indicating better fidelity. Peak Signal-to-Noise Ratio (PSNR) assesses the overall quality by comparing signal strength to noise interference. Fusion Quality Metric ( Q AB / F ) evaluates the effectiveness of feature preservation in the fusion process, while Fusion Noise Metric ( N AB / F ) estimates the level of noise introduced during fusion. Structural Similarity Index (SSIM) measures the similarity in luminance, contrast, and structure between images, whereas Multi-Scale Structural Similarity Index (MS-SSIM) extends SSIM by incorporating multi-scale features for a more comprehensive assessment of perceptual quality. Higher values of VIF, PSNR, Q AB / F , SSIM, and MS-SSIM indicate superior structural preservation and visual fidelity, whereas lower MSE and N AB / F values reflect reduced distortion and minimal noise. These metrics demonstrate the effectiveness and high quality of the fusion process.

4.3. Ablation Study

(1) Sensitivity Analysis of Loss Weights: The total loss is defined as a weighted combination of four sub-loss terms. The initial weights are preset according to the functional roles of the individual loss components, with higher weights assigned to terms that directly supervise structural integrity and chrominance consistency. During the early stage of training, the numerical magnitudes of the loss terms are continuously monitored to maintain scale balance and to prevent any single term from excessively dominating the optimization due to magnitude discrepancies. Furthermore, while keeping the network architecture and all other hyperparameters fixed, each weight is perturbed by multiplying the default value by 0.5 and 2 to conduct a sensitivity analysis of the loss weights. As indicated in Table 1, the overall performance remains relatively stable under these perturbations, while each loss term still influences the fusion behavior in a distinct manner. Increasing the edge-related weight tends to over-constrain high-frequency responses and suppress perceptually important details, whereas reducing this weight weakens the transfer of edge information between modalities. Increasing the pixel-loss weight emphasizes reconstruction accuracy but may bias the optimization toward intensity fitting, thereby slightly degrading perceptual detail integration. The color-balance weight primarily governs chrominance stability: excessively large values may restrict intensity reconstruction, while overly small values make the fusion more susceptible to infrared-induced color shifts. Overall, the default setting achieves the most balanced trade-off across the evaluation metrics and avoids degradation caused by over-emphasizing any single supervision signal. Therefore, this configuration is selected as the final choice to ensure optimization stability and consistent fusion quality.
Table 1. Sensitivity analysis of loss weights on the DroneVehicle dataset. One weight is scaled by × 0.5 or × 2 at a time, while the others are fixed at the default setting ( α 1 , α 2 , α 3 , α 4 ) = ( 10.0 ,   3.0 ,   1.0 ,   5.0 ) . The upward arrow (↑) indicates that a higher value is better, while the downward arrow (↓) indicates that a lower value is better. Bold values indicate the overall optimal results among all compared settings.
(2) 4-Directional Mamba Block: To validate the effectiveness of the MSSA framework, an ablation study is conducted on the DroneVehicle dataset by removing its core component, the 4-Directional Mamba Block, to evaluate its impact on fusion image quality. The experimental results indicate that removing the 4-Directional Mamba Block increases the N AB / F metric while decreasing the PSNR value, which reflects a higher noise level and a reduction in signal quality. Furthermore, the decline in SSIM and MS-SSIM metrics demonstrates a reduced ability of the network to capture features across different scales. This degradation affects structural integrity and introduces texture discontinuities in the fused images, negatively impacting overall visual quality. These findings further confirm the importance of the 4-Directional Mamba Block in multimodal feature extraction and fusion. The removal of this module significantly weakens the capability of the network for global modeling. This change causes noticeable degradation in detail clarity, global consistency, and modality complementarity in the fused images.
(3) Chrominance-Aware: The CMNet incorporates a Chrominance-Aware branching mechanism to enhance the representation of chrominance information in fused images. To validate the effectiveness of this mechanism, an ablation study is conducted on the DroneVehicle dataset to evaluate its impact on model performance. Specifically, a comparison is made between network structures with and without the Chrominance-Aware branch. In the controlled experiment, only the Y channel and the infrared channel are used for training, while the CrCb chrominance channel are bypassed. Concretely, the visible image is converted to the YCrCb space and decomposed into Y, CrCb channels. The network takes the two-channel input [ Y , I IR ] to perform feature extraction and fusion, and predicts only the fused luminance channel Y ^ F . Meanwhile, CrCb channels are not fed into the MSSA framework or any convolutional branch; instead, they are directly skipped to the reconstruction stage via an identity mapping. The final fused color image is then obtained by composing Y ^ F , CrCb channels and converting it back to RGB. During this ablation, all other network components and training settings remain unchanged. As shown in Table 2, the network incorporating the Chrominance-Aware branch outperforms the version without this module across all seven evaluation metrics. Notably, a significant improvement is observed in the VIF metric. This result shows that the branch enhances chrominance representation and improves the retention of fine details in fused images. In contrast to fusion approaches that use only luminance information, this mechanism fully utilizes the chrominance information from visible images. The CrCb channel interact deeply with the fusion features through this design. This mechanism enhances color fidelity and feature representation. It also ensures superior visual consistency and information completeness in the fused images.
Table 2. Component ablation study on DroneVehicle dataset. The upward arrow (↑) indicates that a higher value is better, while the downward arrow (↓) indicates that a lower value is better. The check mark (✓) indicates that the corresponding component or strategy is enabled. The best results are highlighted in red.
(4) CEF: The CMNet incorporates the CEF module. This module facilitates effective chrominance propagation and matching. It achieves chrominance balance across different modalities, reduces spectral distortion, and enhances the visual consistency of fused images. An ablation study is conducted on the DroneVehicle dataset to validate the effectiveness of the CEF module. This study analyzes the impact of removing the CEF module and applying direct additive fusion of the Y and infrared channels, while the CrCb channel are simply concatenated. As shown in Table 2, after removing the CEF module, some low-level texture preservation metrics, such as MSE, are slightly higher than those of the model with the CEF module, which may be attributed to the direct retention of structural information. The fused image shows a significant decline in color consistency in the absence of effective chrominance modeling and balancing. This degradation influences the overall perceptual quality. The VIF metric is considerably lower in the absence of the CEF module. This result reflects the critical role of the module in enhancing visual fidelity. The experiment also shows that additive fusion and channel concatenation cannot fully utilize chrominance information.
(5) Number of Scanning Directions: To analyze the impact of directional serialization on long-range dependency modeling, structural consistency, and overall fusion performance, an ablation study is conducted by varying the number of scanning directions. Notably, the direction setting is applied to both the MSSA framework and the CEF module, since directional serialization is performed in both components. As shown in Table 3, as the number of scanning directions increases, the transfer of edge information from the source images becomes more complete. The structure-related metrics SSIM and Q AB / F exhibit an overall increasing trend. This indicates that multi-directional serialization enables more effective modeling of cross-direction long-range dependencies and enhances the visibility of structural details. In contrast, VIF shows certain fluctuations. This suggests that under direct summation aggregation, high-frequency responses from different directions may introduce local variations and lead to non-uniform enhancement in some regions.
Table 3. Ablation study on the number of scanning directions on the DroneVehicle dataset. Single- and two-direction settings exhibit comparable performance for horizontal and vertical scans, so results are reported by direction count without distinguishing specific scan combinations. The upward arrow (↑) indicates that a higher value is better, while the downward arrow (↓) indicates that a lower value is better. The best results are highlighted in red.

4.4. Comparisons with the Existing Methods

To comprehensively evaluate the performance of the proposed method, quantitative and qualitative comparisons are conducted against nine existing methods on the DroneVehicle, RGBT-Tiny, and VTUAV datasets. The comparison methods include diffusion model-based fusion (Dif-Fusion [27]), deep feature fusion-based approaches (LDRepFM [19], MetaFusion [53], AcFusion [54]), attention mechanism-based methods (AGMFusion [24], FAFusion [21]), a global token modeling method (GTMFuse [55]), a color-fidelity oriented baseline (FCDFusion [47]), and a hybrid attention and cross-modal enhancement method (HAIAFusion [26]).
(1) Quantitative Analysis: Table 4 presents the quantitative results of seven evaluation metrics for 100 image pairs from the DroneVehicle, RGBT-Tiny, and VTUAV datasets. The proposed method achieves superior performance across multiple key metrics. It consistently attains the lowest MSE values in all datasets, reflecting strong pixel-level error control that helps preserve fine details from the source images. The highest PSNR values indicate minimal noise introduction during fusion. This result reflects high-fidelity reconstruction. The superior N AB / F values also confirm excellent noise suppression. These values help avoid excessive distortions in the fused images. In terms of structure retention, the method achieves competitive SSIM and MS-SSIM scores. These scores demonstrate the advantage of the method in maintaining the structural consistency of the source images. High VIF values further validate its ability to preserve crucial image information. This evidence suggests that the method is reliable for multimodal fusion. LDRepFM also performs well in structural preservation, as shown by its strong SSIM and MS-SSIM scores. Its MSE values, however, are not the lowest. This observation suggests that although LDRepFM retains high structural fidelity, its noise suppression remains less effective than that of the proposed method. GTMFuse benefits from transformer-based global modeling and can preserve informative structures, while FCDFusion explicitly stabilizes chrominance; however, both methods still show dataset-dependent trade-offs and fail to maintain consistently balanced performance across the seven metrics.
Table 4. Performance Comparison on RGBT-Tiny, DroneVehicle, and VTUAV Datasets. The upward arrow (↑) indicates that a higher value is better, while the downward arrow (↓) indicates that a lower value is better. The best results are highlighted in red, while the second-best results are highlighted in blue to facilitate comparative analysis.
In contrast, AGMFusion and FAFusion exhibit noticeable weaknesses. Their relatively high MSE values indicate increased reconstruction errors, and their lower PSNR scores suggest inadequate noise minimization. Their poor N AB / F values reflect weaker noise suppression capabilities. Dif-Fusion and AcFusion retain excessive infrared information. This retention results in lower PSNR scores and poor color balance. Their Q AB / F values also suggest a trade-off between feature retention and distortion introduction. MetaFusion achieves competitive PSNR values. However, it suffers from excessive color saturation, which is reflected in its lower Q AB / F values. This issue results in unnatural color distribution in some scenarios. HAIAFusion shows strong performance in VIF and SSIM. These scores demonstrate its ability to preserve fine details. However, its relatively high MSE values reflect a greater degree of reconstruction error compared with the optimal methods. Additionally, its moderate performance in N AB / F indicates that it does not suppress noise as effectively as the proposed method. Overall, the proposed method demonstrates superior performance in detail retention, noise suppression, and structural preservation. Some methods perform well in specific aspects. However, they do not maintain balanced performance across all key indicators. The proposed approach mitigates these trade-offs and produces consistently high-quality fusion results.
The generalization capability of the proposed method is further validated by its strong performance on the RGBT-Tiny and VTUAV datasets, which were not included in the training phase. The model consistently achieves high-quality fusion results across different lighting conditions, weather variations, and environmental complexities. These results demonstrate its adaptability to diverse multimodal fusion scenarios. Some competing methods exhibit performance degradation when applied to unseen datasets. In contrast, the proposed approach maintains robust fusion quality. It preserves both structural and chrominance information across different domains. This stability shows that the model is suitable for real-world applications with dynamic environmental factors. The model generalizes across multiple datasets without fine-tuning. This behavior reflects the effectiveness of the designed feature extraction and fusion strategies. The results confirm the reliability and consistency of the model in multimodal image fusion tasks.
(2) Qualitative Analysis: To validate the fusion performance of the proposed method under different environments, seven image pairs are selected from the DroneVehicle, RGBT-Tiny, and VTUAV datasets, covering diverse scenarios including daytime and nighttime conditions, highways, buildings, rivers, and pedestrians in complex environments. The visual results are shown in Figure 6. An ideal fusion method should retain fine textures from visible light while enhancing the saliency of infrared targets without introducing spectral distortion. AGMFusion and FAFusion suffer from color imbalance. This problem causes overexposure or underexposure in certain areas and results in uneven brightness distribution. Dif-Fusion and AcFusion retain excessive infrared information. This retention leads to visible light color loss, and the fused images often appear overly red or dark. MetaFusion introduces excessive local color saturation. This issue produces unnatural visual effects. GTMFuse can preserve salient structures via global modeling but may still show mild chrominance inconsistency in highlighted regions, while FCDFusion improves hue stability yet may slightly smooth fine textures, revealing a trade-off between color fidelity and detail preservation. HAIAFusion does not sufficiently enhance dark areas in nighttime scenes. As a result, critical target and background details are lost. For example, in the nighttime highway scene, HAIAFusion fails to reveal weak-contrast regions around vehicles, leading to missing background cues, whereas the proposed method maintains clearer lane boundaries and vehicle contours while avoiding chrominance drift.
Figure 6. Visual results on the DroneVehicle, RGBT-Tiny and VTUAV datasets. (a,b) are the source images, (c–k) correspond to nine advanced methods, and (l) represents the proposed method. The red box denotes the zoomed-in region for detailed visualization.
Overall, the proposed method explicitly incorporates a dedicated chrominance branch and an enhanced chrominance fusion mechanism, allowing chrominance information to be consistently perceived, propagated, and regulated throughout the fusion process. This strategy effectively suppresses infrared-induced color shifts while preserving key structural details, achieving a better balance between intensity and chrominance information. The fused result appears more natural and realistic. Meanwhile, a representative success case can be observed in the urban-building scene, where the proposed method preserves sharp edges and fine window structures while keeping pedestrians thermally salient, whereas methods with color imbalance exhibit noticeable local overexposure and saturated highlights. In nighttime scenarios, the model reduces artifacts in background regions. This capability improves visual perception in low-light conditions. The MSSA framework and the CEF module contribute to this performance. These modules extract and integrate multimodal information and support joint optimization of global and local features.

4.5. Time Complexity Analyse

In the complexity analysis, a detailed comparison of computational cost and efficiency is conducted for CMNet and other existing methods. The experimental results indicate that CMNet incorporates the improved Mamba module and a multi-branch architecture. This design enhances the capability of global feature modeling. Meanwhile, the computational efficiency of the network remains at an optimal level. Computational efficiency is a key factor in evaluating model performance. Table 5 presents the average inference time of eight methods on the DroneVehicle, RGBT-Tiny, and VTUAV datasets. Since the image sizes of the three test sets have been standardized, the times reported in the table are computed under the same input resolution.
Table 5. Comparison of average computation time per image (in seconds), parameter size (in thousands, K), and efficiency across different models. The best results are highlighted in red, while the second-best results are highlighted in blue for comparative analysis.
To more precisely measure differences in computational resource consumption across models, the per-parameter computational efficiency (computation time per thousand parameters) is introduced as an evaluation metric. This metric reflects the computational efficiency of a model with respect to its parameter size. It supports a more comprehensive comparison of computational complexity and inference speed across different methods. The Table 5 shows that AGMFusion achieves the best computational speed, with the shortest runtime. This advantage results from its lightweight architecture and minimal parameter count. These characteristics make AGMFusion particularly advantageous in resource-constrained environments. FAFusion has a large number of parameters and incurs the highest computational cost. This situation results in a significantly longer runtime. LDRepFM has a relatively small parameter count. However, it still exhibits a longer computation time. This observation indicates that a reduction in parameter count does not necessarily lead to improved computational speed. Overall network optimization and efficient design remain critical factors.
On the other hand, AcFusion adopts a multi-branch design and attention mechanism. This structure introduces relatively higher computational complexity. As a result, its runtime is slightly longer compared to other methods. Although the proposed method contains a large number of parameters, CMNet still achieves an optimal balance by accelerating processing speed while maintaining computational efficiency. This is attributed to the improved Mamba module, which enhances global modeling capability while reducing computational redundancy. The multi-branch structure in CMNet increases computational complexity. This structure causes its runtime to be slightly higher than that of some lightweight algorithms. At the same time, the design ensures a superior trade-off between computational speed and fusion quality. It also provides strong adaptability across various application scenarios.

4.6. Limitations

As with all complex models, CMNet faces certain challenges when deployed on devices with limited computational resources. These challenges mainly arise from the computational burden introduced by the improved Mamba module and the multi-branch architecture. Although Mamba provides linear complexity and strong global modeling capability, it still has limitations in handling high-frequency details and edge information. These limitations may affect the reconstruction quality in some texture regions. In addition, although the proposed network demonstrates good generalization performance in multimodal image fusion tasks, its performance still requires further improvement under extremely low-light conditions or in scenes involving fast-moving targets. It is important to note that these limitations do not undermine the effectiveness and application potential of CMNet in fusion quality and structural preservation.

5. Conclusions

In this article, CMNet is proposed for aerial remote sensing image fusion. This method first extracts spatial information at multiple scales through the MSSA framework. The network incorporates the Mamba module with linear complexity and achieves efficient and accurate global modeling. The CEF module introduces a chrominance branch to enhance the color representation of the fused image. This design suppresses redundant information and improves texture preservation, difference perception, and cross-modal correlation. The network integrates intensity and chrominance information and captures both structural and detailed features. Experiments across multiple datasets demonstrate that CMNet achieves competitive fusion performance. These results validate the reliability and general applicability of the proposed approach. Future work focuses on optimizing the weighting mechanism to further enhance color fusion performance. It also explores lightweight network design strategies and knowledge distillation techniques to reduce model size and decrease inference time.

Author Contributions

Methodology, S.L.; resources, S.L.; software, S.L.; supervision, S.Z.; writing—original draft, J.C.; writing—review and editing, S.Z. and W.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62575238 and 62202376, in part by the Key Research and Development Program of Shaanxi Province under Grant 2025CY-YBXM-048, and in part by the Scientific Research Plan Project of the Education Department of Shaanxi Province under Grant 24JS048.

Data Availability Statement

Due to privacy, the data cannot be made public.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Li, Q.; Yuan, Y.; Wang, Q. Multiscale Factor Joint Learning for Hyperspectral Image Super-Resolution. IEEE Trans. Geosci. Remote Sens. 2023, 61, 5523110. [Google Scholar] [CrossRef] [Scilit]
  2. Zhang, T.; Li, L.; Cao, S.; Pu, T.; Peng, Z. Attention-Guided Pyramid Context Networks for Detecting Infrared Small Target Under Complex Background. IEEE Trans. Aerosp. Electron. Syst. 2023, 59, 4250–4261. [Google Scholar] [CrossRef] [Scilit]
  3. Li, Q.; Zhang, W.; Lu, W.; Wang, Q. Multibranch Mutual-Guiding Learning for Infrared Small Target Detection. IEEE Trans. Geosci. Remote Sens. 2025, 63, 5605710. [Google Scholar] [CrossRef] [Scilit]
  4. Su, Z.; Zhao, K.; Fan, Z.; Guo, X. RTPV-YOLO: Real-time Photovoltaic Detection with UAV-based Thermal and RGB Imaging. IEEE Trans. Aerosp. Electron. Syst. 2025, 61, 13799–13810. [Google Scholar] [CrossRef] [Scilit]
  5. Ma, J.; Ma, Y.; Li, C. Infrared and visible image fusion methods and applications: A survey. Inf. Fusion 2019, 45, 153–178. [Google Scholar] [CrossRef] [Scilit]
  6. Liu, Y.; Liu, S.; Wang, Z. A general framework for image fusion based on multi-scale transform and sparse representation. Inf. Fusion 2015, 24, 147–164. [Google Scholar] [CrossRef] [Scilit]
  7. Liu, Z.; Feng, Y.; Chen, H.; Jiao, L. A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain. Opt. Lasers Eng. 2017, 97, 71–77. [Google Scholar] [CrossRef] [Scilit]
  8. Yang, B.; Li, S. Multifocus image fusion and restoration with sparse representation. IEEE Trans. Instrum. Meas. 2009, 59, 884–892. [Google Scholar] [CrossRef] [Scilit]
  9. Zhang, S.; Huang, F.; Liu, B.; Li, G.; Chen, Y.; Chen, Y.; Zhou, B.; Wu, D. A multi-modal image fusion framework based on guided filter and sparse representation. Opt. Lasers Eng. 2021, 137, 106354. [Google Scholar] [CrossRef] [Scilit]
  10. Cvejic, N.; Lewis, J.; Bull, D.; Canagarajah, N. Region-based multimodal image fusion using ICA bases. In Proceedings of the 2006 International Conference on Image Processing, Atlanta, GA, USA, 8–11 October 2006; pp. 1801–1804. [Google Scholar]
  11. Ma, J.; Chen, C.; Li, C.; Huang, J. Infrared and visible image fusion via gradient transfer and total variation minimization. Inf. Fusion 2016, 31, 100–109. [Google Scholar] [CrossRef] [Scilit]
  12. Chen, Y.; Cheng, L.; Wu, H.; Mo, F.; Chen, Z. Infrared and visible image fusion based on iterative differential thermal information filter. Opt. Lasers Eng. 2022, 148, 106776. [Google Scholar] [CrossRef] [Scilit]
  13. Ma, J.; Zhou, Z.; Wang, B.; Zong, H. Infrared and visible image fusion based on visual saliency map and weighted least square optimization. Infrared Phys. Technol. 2017, 82, 8–17. [Google Scholar] [CrossRef] [Scilit]
  14. Karim, S.; Tong, G.; Li, J.; Qadir, A.; Farooq, U.; Yu, Y. Current advances and future perspectives of image fusion: A comprehensive review. Inf. Fusion 2023, 90, 185–217. [Google Scholar] [CrossRef] [Scilit]
  15. Li, Q.; Zhang, M.; Yang, Z.; Yuan, Y.; Wang, Q. Edge-Guided Perceptual Network for Infrared Small Target Detection. IEEE Trans. Geosci. Remote Sens. 2024, 62, 5643510. [Google Scholar] [CrossRef] [Scilit]
  16. Zhang, X.; Demiris, Y. Visible and Infrared Image Fusion Using Deep Learning. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 10535–10554. [Google Scholar] [CrossRef]
  17. Li, H.; Wu, X.J. DenseFuse: A Fusion Approach to Infrared and Visible Images. IEEE Trans. Image Process. 2019, 28, 2614–2623. [Google Scholar] [CrossRef] [Scilit]
  18. Raza, A.; Huo, H.; Fang, T. PFAF-Net: Pyramid Feature Network for Multimodal Fusion. IEEE Sens. Lett. 2020, 4, 5501704. [Google Scholar] [CrossRef] [Scilit]
  19. Lu, M.; Jiang, M.; Kong, J.; Tao, X. LDRepFM: A Real-Time End-to-End Visible and Infrared Image Fusion Model Based on Layer Decomposition and Re-Parameterization. IEEE Trans. Instrum. Meas. 2023, 72, 5016012. [Google Scholar] [CrossRef] [Scilit]
  20. Hu, X.; Liu, Y.; Yang, F. PFCFuse: A Poolformer and CNN Fusion Network for Infrared-Visible Image Fusion. IEEE Trans. Instrum. Meas. 2024, 73, 5029714. [Google Scholar] [CrossRef] [Scilit]
  21. Xiao, G.; Tang, Z.; Guo, H.; Yu, J.; Shen, H.T. FAFusion: Learning for Infrared and Visible Image Fusion via Frequency Awareness. IEEE Trans. Instrum. Meas. 2024, 73, 5015011. [Google Scholar] [CrossRef] [Scilit]
  22. Gao, Y.; Ma, S.; Liu, J. DCDR-GAN: A Densely Connected Disentangled Representation Generative Adversarial Network for Infrared and Visible Image Fusion. IEEE Trans. Circuits Syst. Video Technol. 2023, 33, 549–561. [Google Scholar] [CrossRef] [Scilit]
  23. Wu, D.; Han, M.; Yang, Y.; Zhao, S.; Rao, Y.; Li, H.; Lin, X.; Zhou, C.; Bai, H. DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion. IEEE Trans. Instrum. Meas. 2023, 72, 5011815. [Google Scholar] [CrossRef] [Scilit]
  24. Liu, S.; Lan, X.; Chen, W.; Zhang, Z.; Qiu, C. AGMFusion: A Real-Time End-to-End Infrared and Visible Image Fusion Network Based on Adaptive Guidance Module. IEEE Sens. J. 2024, 24, 28338–28350. [Google Scholar] [CrossRef] [Scilit]
  25. Zhang, X.; Wang, X.; Yan, C.; Sun, Q. EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement. IEEE Sens. J. 2024, 24, 4920–4934. [Google Scholar] [CrossRef] [Scilit]
  26. Sun, Y.; Dong, M.; Yu, M.; Zhu, L. HAIAFusion: A Hybrid Attention Illumination-Aware Framework for Infrared and Visible Image Fusion. IEEE Trans. Instrum. Meas. 2025, 74, 5006619. [Google Scholar] [CrossRef] [Scilit]
  27. Yue, J.; Fang, L.; Xia, S.; Deng, Y.; Ma, J. Dif-Fusion: Toward High Color Fidelity in Infrared and Visible Image Fusion With Diffusion Models. IEEE Trans. Image Process. 2023, 32, 5705–5720. [Google Scholar] [CrossRef] [Scilit]
  28. Li, Q.; Yang, Z.; Cheng, J.; Wang, Q. Spatial-Frequency Feature Learning for Infrared Small Target Detection. IEEE Trans. Aerosp. Electron. Syst. 2026. [Google Scholar] [CrossRef] [Scilit]
  29. Li, Q.; Gong, M.; Yuan, Y.; Wang, Q. RGB-Induced Feature Modulation Network for Hyperspectral Image Super-Resolution. IEEE Trans. Geosci. Remote Sens. 2023, 61, 5512611. [Google Scholar] [CrossRef] [Scilit]
  30. Tang, W.; He, F.; Liu, Y.; Duan, Y.; Si, T. DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer. IEEE Trans. Circuits Syst. Video Technol. 2023, 33, 3159–3172. [Google Scholar] [CrossRef] [Scilit]
  31. Li, X.; Li, Y.; Chen, H.; Peng, Y.; Chen, L.; Wang, M. RITFusion: Reinforced Interactive Transformer Network for Infrared and Visible Image Fusion. IEEE Trans. Instrum. Meas. 2024, 73, 5000916. [Google Scholar] [CrossRef] [Scilit]
  32. Gu, A.; Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv 2023, arXiv:2312.00752. [Google Scholar] [CrossRef] [Scilit]
  33. Xu, G.; Liu, Y.; Deng, L.; Wang, X.; Zhu, H. SMNet: A Semantic Guided Mamba Network for Remote Sensing Change Detection. IEEE Trans. Aerosp. Electron. Syst. 2025, 61, 11116–11127. [Google Scholar] [CrossRef] [Scilit]
  34. Li, X.; Chen, R.; Wang, J.; Chen, W.; Zhou, H.; Ma, J. CASPFuse: An Infrared and Visible Image Fusion Method Based on Dual-Cycle Crosswise Awareness and Global Structure-Tensor Preservation. IEEE Trans. Instrum. Meas. 2025, 74, 5002515. [Google Scholar] [CrossRef] [Scilit]
  35. Park, S.; Vien, A.G.; Lee, C. Cross-modal transformers for infrared and visible image fusion. IEEE Trans. Circuits Syst. Video Technol. 2024, 34, 770–785. [Google Scholar] [CrossRef] [Scilit]
  36. Chang, L.; Huang, Y.; Li, Q.; Su, W.; Zhang, Y. L2Net: Infrared and Visible Image Fusion Using Lightweight Large Kernel Convolution Network. IEEE Trans. Instrum. Meas. 2023, 72, 5031713. [Google Scholar] [CrossRef] [Scilit]
  37. Gu, A.; Johnson, I.; Goel, K.; Saab, K.; Dao, T.; Rudra, A.; Ré, C. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Adv. Neural Inf. Process. Syst. 2021, 34, 572–585. [Google Scholar]
  38. Gu, A.; Goel, K.; Ré, C. Efficiently modeling long sequences with structured state spaces. arXiv 2021, arXiv:2111.00396. [Google Scholar]
  39. Smith, J.T.; Warrington, A.; Linderman, S.W. Simplified state space layers for sequence modeling. arXiv 2022, arXiv:2208.04933. [Google Scholar]
  40. Fu, D.Y.; Dao, T.; Saab, K.K.; Thomas, A.W.; Rudra, A.; Ré, C. Hungry hungry hippos: Towards language modeling with state space models. arXiv 2022, arXiv:2212.14052. [Google Scholar]
  41. Zhu, L.; Liao, B.; Zhang, Q.; Wang, X.; Liu, W.; Wang, X. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv 2024, arXiv:2401.09417. [Google Scholar] [CrossRef] [Scilit]
  42. Liu, Y.; Tian, Y.; Zhao, Y.; Yu, H.; Xie, L.; Wang, Y.; Ye, Q.; Liu, Y. Vmamba: Visual state space model. arXiv 2024, arXiv:2401.10166. [Google Scholar]
  43. Liu, C.; Chen, K.; Chen, B.; Zhang, H.; Zou, Z.; Shi, Z. RSCAMA: Remote sensing image change captioning with state space model. IEEE Geosci. Remote Sens. Lett. 2024, 21, 6010405. [Google Scholar] [CrossRef] [Scilit]
  44. Chen, K.; Chen, B.; Liu, C.; Li, W.; Zou, Z.; Shi, Z. RSMamba: Remote sensing image classification with state space model. IEEE Geosci. Remote Sens. Lett. 2024, 21, 8002605. [Google Scholar] [CrossRef] [Scilit]
  45. He, X.; Cao, K.; Zhang, J.; Yan, K.; Wang, Y.; Li, R.; Xie, C.; Hong, D.; Zhou, M. Pan-Mamba: Effective pan-sharpening with state space model. Inf. Fusion 2025, 115, 102779. [Google Scholar] [CrossRef] [Scilit]
  46. Zhuang, C.; Kuang, H.; Wang, H.; Wen, C.; Liu, C.; Yuan, G. PHFuse: Unsupervised color visible and infrared image fusion with preserved hue. Sci. Rep. 2025, 15, 31458. [Google Scholar] [CrossRef] [Scilit]
  47. Li, H.; Fu, Y. FCDFusion: A fast, low color deviation method for fusing visible and infrared image pairs. Comput. Vis. Media 2025, 11, 195–211. [Google Scholar] [CrossRef] [Scilit]
  48. Yao, J.; Zhao, Y.; Bu, Y.; Kong, S.G.; Zhang, X. Color-aware fusion of nighttime infrared and visible images. Eng. Appl. Artif. Intell. 2025, 139, 109521. [Google Scholar] [CrossRef] [Scilit]
  49. Shi, Q.; Xi, Z.; Li, H. Nighttime visible and infrared image fusion based on adversarial learning. Infrared Phys. Technol. 2025, 144, 105618. [Google Scholar] [CrossRef] [Scilit]
  50. Sun, Y.; Cao, B.; Zhu, P.; Hu, Q. Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning. IEEE Trans. Circuits Syst. Video Technol. 2022, 32, 6700–6713. [Google Scholar] [CrossRef] [Scilit]
  51. Ying, X.; Xiao, C.; Li, R.; He, X.; Li, B.; Li, Z.; Wang, Y.; Hu, M.; Xu, Q.; Lin, Z.; et al. Visible-Thermal Tiny Object Detection: A Benchmark Dataset and Baselines. arXiv 2024, arXiv:2406.14482. [Google Scholar] [CrossRef] [Scilit]
  52. Zhang, P.; Zhao, J.; Wang, D.; Lu, H.; Ruan, X. Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022. [Google Scholar]
  53. Zhao, W.; Xie, S.; Zhao, F.; He, Y.; Lu, H. MetaFusion: Infrared and visible image fusion via meta-feature embedding from object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 13955–13965. [Google Scholar] [CrossRef] [Scilit]
  54. Zhu, H.; Wu, H.; He, D.; Lan, R.; Liu, Z.; Pan, X. AcFusion: Infrared and Visible Image Fusion Based on Self-Attention and Convolution With Enhanced Information Extraction. IEEE Trans. Consum. Electron. 2024, 70, 4155–4167. [Google Scholar] [CrossRef] [Scilit]
  55. Mei, L.; Hu, X.; Ye, Z.; Tang, L.; Wang, Y.; Li, D.; Liu, Y.; Hao, X.; Lei, C.; Xu, C.; et al. GTMFuse: Group-attention transformer-driven multiscale dense feature-enhanced network for infrared and visible image fusion. Knowl.-Based Syst. 2024, 293, 111658. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.