SRNet: Spectral Rebalancing Network for Robust RGB–Infrared Object Detection Against Modality Anomalies

Abstract

RGB–infrared object detection relies on complementary information from two modalities, yet it implicitly assumes that the imaging quality of both stays stable. In practice, factors such as component aging and sensor gain drift can degrade one modality, a condition termed a modality anomaly. This work addresses the amplitude-attenuation regime of this condition and evaluates two representative cases: darkening of the visible image and flattening of infrared contrast. Fusion strategies trained on clean data are tied to the healthy statistics of both modalities. Once one side degrades, corrupted features propagate through fixed fusion weights and cause a disproportionate loss in accuracy. To solve this problem, this communication proposes a Spectral Rebalancing Network (SRNet) that restores robustness when either modality is degraded. The network preserves phase and adjusts amplitude in the frequency domain, and it contains two complementary modules: Cross-modal Amplitude Band Swapping (CABS), a training-only augmentation that reduces spectral overfitting at zero inference cost, and Band-wise Spectral Adaptive Fusion (BSAF), which replaces element-wise addition with content-adaptive spectral rebalancing. On M³FD and its degradation benchmarks, SRNet achieves the highest clean-data mAP, and it keeps the best accuracy among the compared methods at every degradation level on both sides. The gap between its accuracy in the two degradation directions is also the smallest.

1. Introduction

Visible and infrared sensors capture complementary information about a scene [1,2,3]. Fusing both modalities enables more accurate and reliable detection than using either alone [4,5,6]. RGB–infrared object detection has thus been widely deployed in autonomous driving and surveillance. These systems increasingly operate in long-term, unattended environments, so their robustness under practical conditions becomes a critical concern [7,8].

Recent work has improved RGB–infrared perception through dynamic and quality-aware fusion. RSDet [9] removes redundant spectral information and dynamically selects useful cross-modal features, while ConTriNet [10] aggregates modality-specific and complementary cues for RGB-T salient object detection. IAF-RTDETR [2] estimates illumination and adjusts multi-scale fusion weights accordingly. These methods adapt fusion using spectral, semantic, or illumination cues. However, they focus on selecting complementary features or on weighting the two modalities according to the scene. None of them models modality degradation in a unified way.

Li et al. [7] use evidential hybrid fusion to estimate uncertainty and improve robustness to modality noise or corruption. For modality degradation, Tian et al. [11] introduce pseudo-degradation training and a quality-aware module. However, as reported in that work, robustness remains imbalanced across the two degradation directions. This imbalance reflects two inherent challenges. First, suppressing the degraded modality discards structural content that may persist after degradation [12,13]. Second, degradations vary widely in spatial appearance, which limits the transferability of designs tailored to a specific form [14,15].

Prior frequency-domain studies show that many common degradations primarily affect the amplitude spectrum in a band-wise manner. The phase spectrum, which encodes structural information such as edges and contours, remains comparatively stable [16,17,18,19]. This commonality suggests that image degradations can be modeled through frequency-domain characteristics rather than spatial appearance, without designing for each spatial form separately. Rather than identifying the form a degradation takes and reweighting the two modalities accordingly, this view compensates for what different degradations share. No degradation-specific supervision or prior is therefore required. We adopt the principle of preserving phase and selectively adjusting amplitude.

Following this principle, we propose a Spectral Rebalancing Network (SRNet) with two complementary modules that model amplitude per band at the input stage and at the feature-fusion stage. At the input stage, Cross-modal Amplitude Band Swapping (CABS) swaps amplitude at randomly selected frequency bands between the two modalities while preserving phase. It reduces spectral overfitting at zero inference cost. However, this training-time regularization cannot adapt the fusion process to the spectral imbalance of a particular test pair. Band-wise Spectral Adaptive Fusion (BSAF) therefore complements CABS at the feature-fusion stage by replacing element-wise addition with content-adaptive, per-band spectral rebalancing. It reads the spectral energy profiles of both modalities and generates band-specific fusion gains to compensate for shifts in the amplitude distribution that vary from sample to sample.

The main contributions of this communication are as follows:

• A spectral rebalancing framework, SRNet, is proposed to improve robustness against amplitude-attenuation modality anomalies in RGB–infrared detection, based on a per-band principle in the frequency domain.
• A training-only band-swapping augmentation, CABS, is introduced to mitigate modality-specific spectral overfitting by exchanging amplitude across the two modalities at zero inference cost.
• A Band-wise Spectral Adaptive Fusion module, BSAF, is proposed to replace fixed element-wise addition and compensate for shifts in the amplitude distribution. It achieves balanced robustness across both degradation directions.

2. Materials and Methods

2.1. Overview

As illustrated in Figure 1, the pipeline consists of three stages. In the data input stage, CABS swaps amplitude bands between the two modalities with probability p during training. This exposes each branch to the other modality’s amplitude statistics at zero inference cost. In the feature-fusion stage, two symmetric CSPDarknet backbones extract visible and infrared features. At each of the P3, P4, and P5 scales, BSAF replaces element-wise addition with band-wise cross-modal fusion. In the multi-scale detection stage, the fused features are aggregated by the PANet neck and fed to three detection heads.

Figure 1. Overview of SRNet. (Top): the dual-stream detection pipeline with training-only CABS augmentation at the input and BSAF fusion at P3–P5. (Bottom): the internal structures of CABS (left) and BSAF (right).

Both CABS and BSAF operate on a shared set of $K=8$ radial frequency bands defined on the normalized frequency radius $\rho (u,v)=(1/\sqrt{2})\sqrt{{(u/u_N)}^2+{(v/v_N)}^2}\in [0,1]$, where $u_N$ and $v_N$ are the Nyquist indices of the transformed map. This gives $\rho =1$ at the spectral corner and $\rho =1/\sqrt{2}$ at the single-axis Nyquist edge. Since $\rho$ is relative to the size of the map, the same band layout applies at every stage. The band edges form a geometric progression, ${\rho }_0=0$ and ${\rho }_k={\rho }_{\min }{(1/{\rho }_{\min })}^{k/K}$ for $k=1,\dots ,K$ with ${\rho }_{\min }=0.01$, so that every band spans an identical frequency ratio. The lowest band also absorbs the DC component. Since natural-image amplitude energy follows the $1/f$ law and concentrates in low frequencies, this constant relative bandwidth prevents the low-frequency range from collapsing into a single band. All band boundaries are softened with sigmoid transitions to suppress Gibbs ringing. Band k is realized as a soft window $W_k\left(\rho \right)=\sigma \left((\rho -{\rho }_{k-1})/{\tau }_k\right)-\sigma \left((\rho -{\rho }_k)/{\tau }_k\right)$, where $\sigma$ is the sigmoid. The outer sides of the first and the last band are set to one, so that no energy is lost at the ends of the spectrum. The transition width is ${\tau }_k=\max \left(0.1({\rho }_k-{\rho }_{k-1}),2/\min (H,W)\right)$. It is proportional to the band width, but it is never narrower than what the spectral grid of that stage can resolve. The coefficient $0.1$ is used for all bands, all stages, and all experiments. The windows are finally normalized at every frequency point, so that ${\sum }_k{W}_k\left(\rho \right)=1$.

2.2. Cross-Modal Amplitude Band Swapping

In a dual-stream architecture, each branch observes only its own modality during training, so it can overfit to the spectral identity of that modality. CABS reduces this overfitting by periodically exposing each branch to the other modality’s amplitude statistics. Because the phase spectrum is preserved during swapping, each branch retains its own structural content and depends less on the amplitude statistics of its own modality. The internal structure of CABS is shown in Figure 1 (bottom left).

Given an aligned visible–infrared image pair, rFFT2 decomposes each image into its amplitude spectrum A and phase spectrum P. For each of the K bands, an independent Bernoulli variable $s_k\sim \mathrm{Bern}\left(q\right)$ determines whether that band is selected for swapping. The selected bands are aggregated into a soft radial mask $M\left(\rho \right)\in [0,1]$, which depends only on frequency coordinates and is shared across all channels. The bidirectional amplitude swap is then computed as

$$\begin{array}{cc}\hfill A_{\mathrm{vis}}^{\prime }& =(1-M)\odot A_{\mathrm{vis}}+M\odot A_{\mathrm{ir}},\hfill \\ \hfill A_{\mathrm{ir}}^{\prime }& =(1-M)\odot A_{\mathrm{ir}}+M\odot A_{\mathrm{vis}},\hfill \end{array}$$

(1)

where ⊙ denotes element-wise multiplication. This exchange is symmetric and preserves information. The component that leaves one side is received by the other rather than discarded, and the phase spectra of both sides remain untouched. The swapped amplitude $A_m^{\prime }$ is recombined with the original phase $P_m$ to form the complex spectrum $S_m^{\prime }=A_m^{\prime }\odot \exp \left(j{P}_m\right)$. This spectrum is then inverse-transformed by irFFT2 and clamped to $[0,1]$ to reconstruct $x_m^{\prime }$, where $m\in \{\mathrm{vis},\mathrm{ir}\}$. CABS is applied after resizing and before mosaic and color augmentations to ensure pixel-level alignment.

2.3. Band-Wise Spectral Adaptive Fusion

Baseline element-wise addition implicitly assumes that both modalities are equally reliable across all frequencies and channels. To enable content-adaptive fusion, BSAF replaces element-wise addition with per-band spectral rebalancing at each fusion scale. Modality degradation can shift the amplitude of the feature spectrum band by band. The fusion process can respond to such shifts and compensate for them once it reads the spectral energy profiles of both modalities, and no degradation-specific supervision is needed. The internal structure of BSAF is shown in Figure 1 (bottom right).

Let $F_m\in {\mathbb{R}}^{B\times C\times H\times W}$ (batch, channel, height, width) denote the feature map of modality $m\in \{\mathrm{vis},\mathrm{ir}\}$. During training, $F_m$ is extracted from the CABS-augmented input, whereas at inference it is extracted from the original input. The feature spectrum is obtained as $S_m=\mathrm{rFFT2}\left(F_m\right)\in {\mathbb{C}}^{B\times C\times H\times (\lfloor W/2\rfloor +1)}$, with amplitude $A_m=\left|S_m\right|$. This amplitude is distinct from the image-level swapped amplitudes $A_{\mathrm{vis}}^{\prime }$ and $A_{\mathrm{ir}}^{\prime }$ in (1). To characterize the spectral distribution of each modality, a band descriptor $E_m$ is constructed from two complementary components: a band-energy descriptor and a shape descriptor. The band-energy descriptor captures the per-band magnitude:

$$E_m^{\mathrm{eng}}\left[k\right]=\log \left(1+\sum _{(u,v)\in {\mathrm{\Omega }}_k}{A}_m[u,v]\right),$$

(2)

where ${\mathrm{\Omega }}_k$ is the set of frequency points belonging to the kth band. Each frequency is assigned to the single band whose edges bracket its radius, so that the bands form a disjoint partition of the spectrum and every frequency is counted once. The descriptor sums amplitudes rather than squared magnitudes, which keeps it on the same scale as the gains in (6). The bands contain different numbers of frequency points, but $|{\mathrm{\Omega }}_k|$ is fixed for all inputs at a given resolution. In the logarithmic domain it therefore contributes only an offset that does not depend on the input, and this offset is absorbed by the bias of the first gating layer. The descriptor is read to compare the same band across modalities and across samples, not to compare different bands against each other. A band that contains no sampled frequency at a given resolution yields $E_m^{\mathrm{eng}}\left[k\right]=0$ for every input and is inert for the same reason. The shape descriptor normalizes the energy vector to encode distributional form, so that the subsequent gating can read both the amount and the shape of spectral energy:

$$E_m^{\mathrm{shp}}={\displaystyle \frac{E_m^{\mathrm{eng}}}{{\sum }_k{E}_m^{\mathrm{eng}}\left[k\right]+ϵ}},E_m=[E_m^{\mathrm{eng}};E_m^{\mathrm{shp}}],$$

(3)

where $ϵ$ is a small constant for numerical stability, and $[·;·]$ denotes concatenation. The shape descriptor also removes any dependence on the overall scale. The descriptors enter the gating pathway with stop-gradient to prevent gating gradients from interfering with the main feature pathway.

The descriptors from both modalities are concatenated and passed through a joint multilayer perceptron (MLP), denoted ${\mathrm{MLP}}_g$, to produce per-band fusion gains $G_m\in {\mathbb{R}}^K$:

$$[G_{\mathrm{vis}};G_{\mathrm{ir}}]=0.5+1.5\sigma \left({\mathrm{MLP}}_g\left([E_{\mathrm{vis}};E_{\mathrm{ir}}]\right)\right),$$

(4)

where $\sigma$ denotes the sigmoid function. The coefficient $1.5$ together with the offset $0.5$ maps each gain to $[0.5,2.0]$. This range allows both attenuation and amplification by a factor of up to two, and it prevents unstable feature scaling. In addition, two modality-specific MLPs generate cross-modulation terms $T_m\in {\mathbb{R}}^K$ that capture inter-modal interactions:

$$T_m=0.5+1.5\sigma \left({\mathrm{MLP}}_x^m\left([E_{\mathrm{vis}};E_{\mathrm{ir}}]\right)\right),$$

(5)

where ${\mathrm{MLP}}_x^{\mathrm{vis}}$ and ${\mathrm{MLP}}_x^{\mathrm{ir}}$ have separate parameters. Although $G_m$ and $T_m$ are predicted from the same descriptor, they are used differently. $G_m$ is a joint allocation shared by both pathways. It enters the spectral gains through the product $G_m\odot T_m$, and it is separately averaged into the scalar $w_m$ that scales the spatial pathway in (9). $T_m$ acts only on the spectral side, where it refines the shared allocation for each modality. A single output predicting their product is therefore not an equivalent choice, because it would leave the spatial pathway without a gating signal. The K-dimensional band gains are then expanded to full spectral resolution through the normalized soft windows $W_k$ defined in Section 2.1, producing the per-frequency gain map:

$${\mathrm{gain}}_m={\displaystyle \sum _{k=1}^K}(G_m\odot T_m)\left[k\right]W_k.$$

(6)

The resulting real-valued gains modulate the complex spectra of both modalities, and the fused spectrum is transformed back to the spatial domain:

$$S_{\mathrm{fuse}}={\mathrm{gain}}_{\mathrm{vis}}\odot S_{\mathrm{vis}}+{\mathrm{gain}}_{\mathrm{ir}}\odot S_{\mathrm{ir}},$$

(7)

$$Y=\mathrm{irFFT2}\left(S_{\mathrm{fuse}}\right),$$

(8)

where Y denotes the spatially reconstructed fused feature. The final fused feature is obtained as

$$F_{\mathrm{fuse}}=Y+(F_{\mathrm{vis}}+F_{\mathrm{ir}})+\beta \mathrm{CM}\left([w_{\mathrm{vis}}{F}_{\mathrm{vis}};w_{\mathrm{ir}}{F}_{\mathrm{ir}}]\right),$$

(9)

where $w_m=(1/K){\sum }_k{G}_m\left[k\right]$ is a scalar summary of the band gains of modality m. The term $F_{\mathrm{vis}}+F_{\mathrm{ir}}$ serves as a fixed residual base that preserves a baseline-addition anchor for training stability. CM denotes a spatial pathway that maps the concatenated input $[w_{\mathrm{vis}}{F}_{\mathrm{vis}};w_{\mathrm{ir}}{F}_{\mathrm{ir}}]\in {\mathbb{R}}^{B\times 2C\times H\times W}$ back to ${\mathbb{R}}^{B\times C\times H\times W}$. It supplements cross-channel interactions beyond the pointwise spectral gains. A $1\times 1$ convolution reduces the channel count from $2C$ to C, followed by BN and SiLU, and a $3\times 3$ convolution with stride 1 and padding 1 preserves both the channel count and the spatial size. The input of CM is first scaled by $w_m$, so that the spatial pathway follows the same gating decision as the spectral pathway rather than acting independently. Since $Y=\mathrm{irFFT2}\left(S_{\mathrm{fuse}}\right)$ is restored to ${\mathbb{R}}^{B\times C\times H\times W}$, all three terms in (9) share the same shape. All MLP and CM output layers are zero-initialized, so the initial gains are constants that do not depend on the input. The spatial pathway therefore contributes nothing at the start of training, whatever the value of the learnable scalar $\beta$. BSAF thus starts as a constant-scaled addition whose scaling is absorbed by subsequent batch normalization, and adaptive behavior grows smoothly during training.

2.4. Experimental Setup

2.4.1. Datasets

M³FD [20] contains 4200 aligned visible–infrared image pairs captured by a bispectral camera, with annotations for six classes: person, car, bus, lamp, motorcycle, and truck. The dataset is split into 3360 training and 840 validation pairs.

The two degradation sets instantiate the amplitude-attenuation regime defined in Section 1. They are intended as controlled, reproducible probes of that regime.

M³FD-DV (visible degraded, infrared clean) applies three levels of low-light degradation [21,22] to the visible side of the 840 validation pairs through gamma compression with linear gain reduction, $x^{\prime }=x^{\gamma }g$, where x is the 8-bit intensity normalized to $[0,1]$. The mapping is applied identically to the three color channels, and the result is clipped to $[0,1]$, rescaled to $[0,255]$, and quantized to 8 bits. The three levels are mild ($\gamma =1.6$, $g=0.55$), moderate ($\gamma =1.9$, $g=0.45$), and severe ($\gamma =2.5$, $g=0.28$).

M³FD-DI (infrared degraded, visible clean) applies three levels of thermal-contrast degradation [23,24] to the infrared side through high-value clipping, thermal-pedestal addition, and contrast compression, applied in this order to the single intensity channel $x\in [0,255]$. Let $t_P$ be the P-th percentile of the intensity histogram of that image. The three steps are $x_1=\min (x,t_P)$, $x_2=x_1+b$, and $x^{\prime }=\mu +c(x_2-\mu )$, where $\mu$ is the mean of $x_2$, that is, of the map that has already been clipped and offset. The percentage P denotes the percentile above which intensities are truncated, so that the hottest $(100-P)\%$ of the pixels are collapsed onto a common ceiling. The result is clipped to $[0,255]$ and quantized to 8 bits. The three levels are mild ($P=98\%$, $c=0.80$, $b=8$), moderate ($P=92\%$, $c=0.50$, $b=26$), and severe ($P=85\%$, $c=0.30$, $b=38$).

Both amplitude-based degradation sets are used only for evaluation and are never seen during training. Representative examples are shown in Figure 2.

Figure 2. The three-level degradation protocol (mild/moderate/severe) applied to a sample pair from M³FD, with low-light degradation for the visible modality and thermal-contrast degradation for the infrared modality.

2.4.2. Implementation Details

The network is trained for 500 epochs using SGD with an initial learning rate of 0.01 (cosine decay to $0.2\times$), a batch size of 16, and an input resolution of $640\times 640$. All experiments are conducted on a single NVIDIA RTX 4090. Standard YOLOv5s augmentations are applied. For fair comparison, all CSPDarknet53-based competing methods are unified to the small backbone variant and trained with the same schedule as SRNet. Their remaining hyperparameters are kept at the defaults of their respective official implementations. For CABS, p and q are both set to 0.3. For BSAF, the learnable scalar $\beta$ is initialized to 0.2 and updated together with the network parameters. This value is used in all experiments.

3. Results and Discussion

3.1. Comparison with State-of-the-Art Methods

Table 1 compares SRNet with recent methods on the clean M³FD validation set. SRNet achieves the highest mAP and mAP50, and it ranks second on mAP75. Although the margins are modest, they confirm that the robustness mechanisms of SRNet incur no cost in clean-data accuracy.

Table 1. Comparison with state-of-the-art methods on clean M³FD. Bold indicates the highest precision.

Model	Modality	Backbone	mAP50	mAP75	mAP
YOLOv5s	VIS	CSPDarknet53	83.6	54.1	52.1
YOLOv5s	IR	CSPDarknet53	81.4	52.9	51.4
TINet [8]	VIS + IR	ResNet50	79.8	53.4	49.5
RSDet [9]	VIS + IR	ResNet50	86.6	60.2	55.3
ReFusion [25]	VIS + IR	CSPDarknet53	84.9	57.9	54.2
ICAFusion [26]	VIS + IR	CSPDarknet53	86.0	59.2	55.9
${\mathrm{EI}}^2$Det [27]	VIS + IR	CSPDarknet53	85.7	59.5	56.1
CDC-YOLO [28]	VIS + IR	CSPDarknet53	85.4	59.1	55.2
Ours	VIS + IR	CSPDarknet53	86.8	59.6	56.2

Table 2 and Table 3 report mAP under visible-side and infrared-side degradation, respectively. SRNet achieves the best performance at every degradation level on both sides. The directional gap, defined as the difference between the severe scores of the two sides, is only 1.1 points for SRNet and 1.3 points for CDC-YOLO, while the other three methods show gaps of 9.8 to 18.3 points. Both balanced methods perform cross-modal mixing at the input stage to prevent the fusion backbone from binding to one modality’s appearance. CDC-YOLO swaps random spatial patches, which mixes both content and appearance at a regional level. SRNet instead exchanges amplitude at selected frequency bands while preserving phase. It therefore targets only the spectral statistics, and it exceeds CDC-YOLO by 2.1 and 2.3 points of mAP on the two severe settings.

Table 2. Robustness comparison under visible-modality degradation (M³FD-DV). Bold indicates the highest precision.

Model	Clean	Mild	Moderate	Severe
YOLOv5s (VIS)	52.1	50.3	48.1	36.5
RSDet [9]	55.3	47.3	40.4	26.7
${\mathrm{EI}}^2$Det [27]	56.1	54.6	53.3	46.7
ICAFusion [26]	55.9	54.3	52.6	47.2
CDC-YOLO [28]	55.2	54.5	53.7	49.8
Ours	56.2	55.7	55.1	51.9

Table 3. Robustness comparison under infrared-modality degradation (M³FD-DI). Bold indicates the highest precision.

Model	Clean	Mild	Moderate	Severe
YOLOv5s (IR)	51.4	50.1	42.8	12.0
RSDet [9]	55.3	53.8	51.0	45.0
${\mathrm{EI}}^2$Det [27]	56.1	55.1	51.3	35.7
ICAFusion [26]	55.9	54.8	51.6	37.4
CDC-YOLO [28]	55.2	55.1	53.6	48.5
Ours	56.2	55.8	54.8	50.8

The single-modality rows of the three tables show what fusion contributes. On clean data the visible-only detector is competitive at mAP50 but falls behind by 5.5 and 4.1 points on mAP75 and mAP, where localization quality dominates. The gap widens once a modality degrades. With no alternative input, the two baselines drop to 36.5 and 12.0 under severe degradation, against 51.9 and 50.8 for SRNet.

On DV-severe, SRNet surpasses the clean IR-only detector, which shows that a severely darkened visible input still contributes rather than only contaminating the fusion. On DI-severe, a gap remains because thermal-contrast collapse destroys target-background separation irreversibly, whereas low-light darkening preserves recoverable intensities. Closing this gap is left for future work.

3.2. Ablation Study

Table 4 reports the contributions of CABS and BSAF. As a zero-cost regularizer, CABS alone improves DV-severe and DI-severe mAP by 4.2 and 11.8 points with no change in parameters or FLOPs. BSAF alone improves the baseline by 1.8 and 2.1 points, and it further lifts CABS to the best results. To isolate band granularity, BSAF is reduced to a single band ($K=1$), which keeps parameters, FLOPs, and inference speed unchanged. This variant recovers almost none of the gain and barely exceeds CABS alone, so the improvement comes from per-band resolution rather than from added capacity. The FPS column confirms that CABS is free at inference. BSAF costs 5.3% of the throughput, which is well below the 25.5% increase in FLOPs, because the spectral transforms parallelize efficiently on GPU. At 175.4 FPS the full model remains well within real-time operation. The two modules are thus complementary. CABS gives cost-free robustness from the input, and BSAF refines fusion at the feature level.

Table 4. Ablation study and model complexity. DV-Sev. and DI-Sev. denote the severe degradation level of M³FD-DV and M³FD-DI, respectively. Bold indicates the highest precision.

CABS	BSAF	DV-Sev. mAP	DI-Sev. mAP	Params (M)	FLOPs (G)	FPS
×	×	47.0	37.9	11.27	27.20	185.2
✓	×	51.2	49.7	11.27	27.20	185.2
×	✓	48.8	40.0	15.08	34.14	175.4
✓	✓ (K = 1)	51.3	49.9	15.08	34.14	175.4
✓	✓	51.9	50.8	15.08	34.14	175.4

Table 5 reports the design choices of BSAF, with one factor changed at a time. The number of bands shows a single peak at $K=8$. Accuracy rises from $K=1$ to $K=8$ and drops slightly at $K=16$. All four settings share the same parameter count, so the drop reflects the smaller number of frequency samples that fall in each band rather than any change in capacity. Replacing the geometric spacing with a linear one costs 0.5 and 0.8 points. Linear spacing merges the low-frequency range into a single band and leaves the remaining bands unable to separate the degradation signatures shown in Figure 4. Removing the stop-gradient lowers accuracy by 0.8 and 0.5 points, which indicates that gating gradients do interfere with the main feature pathway when allowed to propagate. Restricting the fusion to P3 is the most damaging configuration. Its parameter count falls accordingly, so the drop reflects both the narrower coverage and the reduced capacity. The result still confirms that rebalancing is needed at all three scales rather than at the highest resolution alone.

Table 5. Design choices of BSAF. Each row changes one factor with respect to the full configuration in the last row. Bold indicates the full configuration and the highest precision.

K	Boundary	Stop-Grad.	Fusion Scales	DV-Sev. mAP	DI-Sev. mAP	Params (M)
1	Geometric	✓	P3–P5	51.3	49.9	15.08
4	Geometric	✓	P3–P5	51.5	50.6	15.08
16	Geometric	✓	P3–P5	51.6	50.6	15.08
8	Linear	✓	P3–P5	51.4	50.0	15.08
8	Geometric	×	P3–P5	51.1	50.3	15.08
8	Geometric	✓	P3 only	50.2	49.3	11.47
8	Geometric	✓	P3–P5	51.9	50.8	15.08

3.3. Spectral Rebalancing Analysis

Figure 3 shows how BSAF responds when one modality is degraded at inference, using the per-band gains at the P4 scale. The horizontal axis is the normalized radial frequency $\rho$. The vertical axis plots the ratio ${\overline{g}}_k^{\mathrm{deg}}/{\overline{g}}_k^{\mathrm{cln}}$ between the degraded-input and clean-input gain of the degraded branch, so a value above one marks a band whose gain is raised. Both curves lie above one, which shows that BSAF lifts the gain of the degraded branch to restore amplitude energy for the corrupted modality. Since the phase spectrum stays intact, the structural content of that modality remains usable and worth amplifying rather than discarding. The increase varies across bands rather than being uniform, which echoes the ablation where a single band ($K=1$) recovers almost none of the benefit. At the P4 resolution the first band contains only the DC component and the second contains no sampled frequency, so both are omitted.

Figure 3. Per-band gain ratio ${\overline{g}}_k^{\mathrm{deg}}/{\overline{g}}_k^{\mathrm{cln}}$ of the degraded branch under severe single-modality degradation (P4 scale).

To characterize the two degradations beyond their spatial appearance, their effect on the amplitude spectrum is measured across frequency bands. For every band k, the ratio $E_k^{\mathrm{deg}}/E_k^{\mathrm{cln}}$ between the degraded and clean band energy is computed per image and averaged (geometric mean) over the 840 validation pairs. The top axis maps each radial frequency $\rho$ to a spatial wavelength $\lambda =1/\left(0.7071\rho \right)$ in pixels. As shown in Figure 4, both degradations attenuate amplitude energy at every band, which confirms that they are amplitude-domain corruptions. Their band-wise signatures, however, differ markedly. Low-light degradation scales the spectrum almost uniformly and leaves a nearly flat profile, consistent with a monotonic global mapping. Thermal-contrast degradation instead removes energy selectively, with the deepest loss concentrated around $\lambda \approx 30$ px, which is the scale of pedestrians and vehicles. A single global gain cannot match two dissimilar band profiles at once, so compensating for such distribution shifts requires adjusting the fusion per band rather than globally. This is the design granularity adopted by SRNet. The outermost band ($\lambda \approx 2$ px) is excluded, as it approaches the Nyquist limit and the quantization noise floor.

Figure 4. Per-band amplitude energy ratio $E_k^{\mathrm{deg}}/E_k^{\mathrm{cln}}$ between clean and severely degraded inputs, averaged over the validation set (shaded: $\pm 1$ std).

Figure 4. Per-band amplitude energy ratio $E_k^{\mathrm{deg}}/E_k^{\mathrm{cln}}$ between clean and severely degraded inputs, averaged over the validation set (shaded: $\pm 1$ std).

3.4. Qualitative Comparison and Analysis

Figure 5 shows the two modules under severe degradation. The third and fourth columns show the CABS output. The exchanged bands introduce visible texture, yet the buildings, the pylon, and the vehicles remain in place, because the phase spectrum is left untouched. The last two columns compare the fused response at the P4 scale, the same stage analyzed in Figure 3. Under DV-severe, where the visible image is almost entirely dark, the baseline response is scattered and weak, whereas BSAF recovers the pedestrian and the vehicle line. Under DI-severe, where infrared contrast has collapsed, BSAF likewise produces a stronger and more concentrated response on the vehicles. The per-band gains reported in Figure 3 thus translate into a visibly restored spatial response for the degraded branch.

Figure 5. Visualization of the two modules under severe degradation on a sample pair from M³FD. Columns 3–4 show the CABS output, where amplitude bands are exchanged and phase is preserved. Columns 5–6 compare the Grad-CAM response of the fused features at the P4 scale for element-wise addition and BSAF.

Figure 6 presents detection results under severe degradation on M³FD-DV (top row) and M³FD-DI (bottom row), with the degraded inputs shown in the first two columns. In the DV scene, the visible image is heavily darkened, so the infrared modality becomes the primary information source. ICAFusion produces a false detection on the road area (yellow arrow). ${\mathrm{EI}}^2$Det avoids this false alarm but misses several objects (yellow dashed circles). SRNet detects the objects missed by the compared methods without introducing false positives. In the DI scene, the infrared image suffers severe contrast loss, which makes it difficult to distinguish thermal signatures from the background. Both ICAFusion and ${\mathrm{EI}}^2$Det exhibit missed detections. They fail to locate objects whose thermal signatures have collapsed. SRNet produces more complete detections with fewer errors, and it is therefore robust regardless of which modality is degraded.

Figure 6. Qualitative comparison under severe degradation on M³FD-DV (degraded visible, clean infrared) (top) and M³FD-DI (degraded infrared, clean visible) (bottom). Yellow arrows indicate false detections and yellow dashed circles indicate missed detections. Detection boxes are overlaid on the degraded modality for visualization.

4. Conclusions

This communication addresses the amplitude-domain modality anomaly in RGB–infrared object detection. Since degradation mainly affects the amplitude spectrum while phase largely preserves content structure [16,17,18,19], we preserve phase and adjust amplitude per band at both ends of the network: CABS, a training-only amplitude-swapping augmentation with zero inference cost, and BSAF, a band-wise spectral rebalancing fusion module. On M³FD and its degradation benchmarks, SRNet achieves the best clean-data mAP and the best robustness among the compared methods across both degradation directions.

The premise of this design is that a degradation attenuates amplitude while it leaves phase intact. The evaluation is limited to two degradations that satisfy it, and the results are not evidence of robustness to sensor anomalies in general. Corruptions that disturb the phase spectrum lie outside the premise altogether. The method itself, however, is supervised by no particular degradation. The two cases studied here have markedly different band-wise signatures (Figure 4) yet benefit from the same mechanism. How far this transfers is left for future work, together with two further directions: extending the shared band gains to spatially varying ones, and handling phase-domain corruptions.

We hope this work draws attention to the amplitude-domain modality anomaly as a practical yet underexplored failure mode in multispectral perception.