UMFNet: Frequency-Guided Multi-Scale Fusion with Dynamic Noise Suppression for Robust Low-Light Object Detection

Abstract

The dominant low-light object detectors face the following spectral trilemma: (1) the loss of high-frequency structural details during denoising, (2) the amplification of low-frequency illumination distortion, and (3) cross-band interference in multi-scale features. To resolve these intertwined challenges, we present UMFNet—a frequency-guided detection framework that unifies adaptive frequency distillation with inter-band attention coordination. Our technical breakthroughs manifest through three key innovations: (1) a frequency-adaptive fusion (FAF) module employing learnable wavelet kernels (16–64 decomposition basis) with dynamic SNR-gated thresholding, achieving an 89.7% photon utilization rate in ≤1 lux conditions—2.4× higher than fixed-basis approaches; (2) a spatial-channel coordinated attention (SCCA) mechanism with dual-domain nonlinear gating that reduces high-frequency hallucination by 37% through parametric phase alignment (verified via gradient direction alignment coefficient ${\rho }_G$ = 0.93); (3) a spectral perception loss combining the frequency-weighted structural similarity index measure (SSIM) with gradient-aware focal modulation, enforcing physics-constrained feature recovery. Extensive validation demonstrates UMFNet’s leadership: 73.1% mAP@50 on EXDark (+6.4% over YOLOv8 baseline), 58.7% on DarkFace (+3.1% over GLARE), and 40.2% on thermal FLIR ADAS (+9.7% improvement). This work pioneers a new paradigm for precision-critical vision systems in photon-starved environments.

1. Introduction

Modern autonomous systems demand reliable object detection under extreme low-light conditions—a realm where even state-of-the-art detectors fall significantly short. As per the EXDark benchmark [1], YOLOv8 suffers a catastrophic drop when the illumination falls below 5 lux. This performance drop stems from an unresolved spectral trilemma: (1) aggressive high-frequency (HF) denoising erodes critical edges, (2) low-frequency (LF) enhancement amplifies illumination bias, and (3) cross-band interference corrupts multi-scale feature hierarchies.

Prior works navigate this minefield through two flawed paradigms.

Enhance-then-Detect (EtD) approaches [2,3] apply frequency-blind enhancement before detection. Although computationally efficient, their fixed operations over-smooth HF details—DarkFace evaluations [4] reveal 11.2% lower edge accuracy versus daylight performance.

Joint Enhancement–Detection (JED) frameworks [5] co-train enhancement and detection modules. However, without explicit frequency disentanglement, noisy LF components propagate unrestricted, limiting their AP@50 to 31.8% at ≤1 lux—which is inadequate for safety-critical applications.

The core limitation is static frequency processing: existing methods employ rigid wavelet discrete cosine transform (DCT) bases that are incapable of adapting to dynamic noise distributions. As the illuminance decreases, the HF noise variance grows exponentially, while the LF signals shift nonlinearly due to Bayer demosaicing artifacts. Treating these intrinsically coupled phenomena as separate problems proves fundamentally inadequate. We challenge this status quo through UMFNet—the first architecture unifying three breakthroughs. Frequency-Adaptive Fusion (FAF): We replace fixed filters with learnable convolutional wavelets that dynamically adjust the HF/LF thresholds via detection loss gradients. Spatial-Channel Coordinated Attention (SCCA): We apply cross-band gates where LF semantics guide HF denoising. Spectral Perception Loss: We jointly optimize the frequency-weighted SSIM and detection confidence gradients, enforcing physics-informed feature recovery. Extensive validation confirms UMFNet’s leadership: 73.1% mAP@50 on EXDark (+3.4% over SOTA). Crucially, our frequency gradient alignment principle establishes a new theoretical foundation for vision systems in photon-starved environments. Furthermore, compared with the original YOLOv8, our method improves the mean average precision at a 50% intersection over union (mAP50) by approximately 6.4%. This indicates that the combination of UMFNet and YOLOv8 significantly enhances the detection accuracy under low-light conditions. The remainder of this paper is organized as follows. Section 2 reviews related work in low-light image enhancement and object detection. Section 3 presents our proposed UMFNet method in detail. Section 4 describes the experimental setup and results. Finally, Section 5 concludes our work and suggests future research directions. Based on the limitations identified in prior works, we establish three explicit research objectives.

• Objective 1: To develop a frequency-adaptive fusion mechanism that dynamically adjusts the high-/low-frequency thresholds based on the illumination conditions, addressing the static processing limitation in existing methods.
• Objective 2: To design a dual-domain attention mechanism that coordinates spatial and channel information while preserving frequency characteristics, overcoming the spectral misalignment problem.
• Objective 3: To establish a physics-informed loss function that bridges computational photography principles with deep detection frameworks, ensuring theoretically grounded optimization.

2. Related Work

2.1. Methods for Low-Light Image Enhancement

Low-light image enhancement (LLIE) aims to restore image details and reduce color shifts to improve the performance in computer vision tasks. Early methods were primarily based on traditional image processing techniques, such as Retinex theory [6] and histogram equalization [7]. While these methods can improve the brightness and contrast, they are limited in handling details, noise suppression, and color restoration. In recent years, deep learning has advanced LLIE. Convolutional neural network (CNN)-based methods [6,7,8,9], such as KinD [10], restore details through adaptive network structures without requiring paired data. Zero-DCE [3] designs non-reference loss functions, eliminating the need for paired data and achieving excellent performance. Generative adversarial network (GAN) methods [11], like the unsupervised GAN by Jiang et al., have also gained widespread attention by restoring low-light images from unlabeled data through self-supervised learning. However, although low-light enhancement methods make images appear brighter and clearer to the naked eye, they still leave a lot of background noise, thus having a minimal impact in terms of improving the detection performance in low-light object detection.

2.2. Methods for Low-Light Object Detection

To enhance the detection performance, recent studies have attempted to jointly perform image enhancement and object detection. IA-YOLO [12] combines image enhancement with YOLO detection by using a CNN-based parameter predictor to automatically adjust the enhancement module parameters, optimizing low-light image enhancement and improving the detection capabilities. Although IA-YOLO performs well, it relies on a specific enhancement module, making it difficult to adapt to diverse low-light conditions in complex environments. MAET [13] optimizes the image restoration process through an image signal processing (ISP) pipeline based on a physical noise model. It uses an adaptive neural network to predict the degradation parameters and avoids feature mixing through regularization, significantly enhancing the detection performance under low-light conditions. However, despite the detailed description and parameterization of the ISP pipeline by Cui et al. [13], the selection of these parameters is based on certain fixed ranges and does not take into account the dynamically changing factors in the actual shooting process, such as automatic white balance adjustment and exposure compensation.

2.3. Joint Enhancement and Detection Methods

Joint enhancement and detection (JED) methods excel in low-light object detection by simultaneously optimizing the image quality and detection accuracy during training. Deyolo [14] employs an end-to-end joint optimization framework, integrating multi-scale enhancement and detection modules to improve YOLO’s [15,16,17] performance and robustness in low-light environments. PE-YOLO [18] also employs a similar approach. In simple terms, this cascaded method is plug-and-play and end-to-end, capable of enhancing features while removing noise. In the experimental section of this paper, we also use this method to evaluate the performance of our approach.

3. Method

UMFNet is a frequency-guided enhancement framework specifically designed for object detection in low-light conditions, aiming to solve the “spectral trilemma” described above: (1) the loss of high-frequency structural details during the denoising process; (2) the amplification of low-frequency illumination distortion; and (3) the cross-band interference in multi-scale features. As shown in Figure 1, UMFNet is connected as a pre-enhancement module before the YOLOv8 detector to form an end-to-end trainable system.

Given a low-light input image $I\in {\mathbb{R}}^{3\times H\times W}$, our goal is to learn the mapping $F_{\theta }:I\mapsto I_{enhanced}$ to generate an enhanced image, which is then fed into YOLOv8 for detection. UMFNet mainly consists of the following parts.

Encoder: It receives the low-light input image and generates an initial feature representation.

Residual Blocks ($\times \mathbf{2}$): They extract and refine multi-level features.

Frequency-Adaptive Fusion (FAF) Module: It processes different frequency components to achieve illumination-adaptive frequency-domain decomposition and fusion.

Spatial-Channel Coordinated Attention (SCCA) Mechanism: It coordinates the interactions between frequency-domain features, reduces high-frequency hallucinations, and enhances key structural information.

Decoder: It reconstructs the processed features into an enhanced image for further processing by YOLOv8.

3.1. Frequency-Adaptive Fusion Module (FAF)

The frequency-adaptive fusion (FAF) module is one of the core innovations of UMFNet. It addresses the limitations of static frequency processing in low-light object detection. Traditional multi-scale fusion methods [19] apply fixed filtering operations regardless of the input conditions, resulting in poor performance under complex illumination conditions. FAF overcomes this limitation through steerable spectral decomposition and attention-driven recombination. The FAF module is shown in the Figure 2.

3.1.1. Mathematical Model

The design of the FAF module is based on a key insight: when the illuminance decreases, the variance of high-frequency noise increases exponentially, while the low-frequency signals shift nonlinearly due to Bayer demosaicing artifacts. Our method treats these intrinsically coupled phenomena as a unified problem.

Steerable Spectral Decomposition. Given multi-scale features $F_{hr}\in {\mathbb{R}}^{C_{hr}\times H\times W}$ and $F_{lr}\in {\mathbb{R}}^{C_{lr}\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}}$, our adaptive decomposition learns the importance of frequency bands:

$$\begin{array}{cc}\hfill {\mathcal{H}}_{high}\left(F\right)& =F\ast \mathrm{LoG}\left({\sigma }_t\left(l\right)\right)\hfill \\ \hfill {\mathcal{H}}_{low}\left(F\right)& =\mathrm{GaussianBlur}(F,\mathrm{Adaptive}\Sigma (l\left)\right)\hfill \end{array}$$

(1)

where LoG is the Laplacian-of-Gaussian operator, which is used to extract high-frequency components; GaussianBlur applies low-pass filtering; and $\mathrm{Adaptive}\Sigma \left(l\right)$ determines the size of the blur kernel based on the illumination level l.

The scale parameter ${\sigma }_t\left(l\right)={\alpha }_t·e^{-{\beta }_t·l}$ decays exponentially as the illumination decreases, modeling the physical behavior of photon-dependent noise. The parameters ${\alpha }_t$ and ${\beta }_t$ are calculated as follows:

$$\begin{array}{cc}\hfill {\alpha }_t& ={\alpha }_{base}+W_{\alpha }·F_{\theta }\left(I\right)\hfill \\ \hfill {\beta }_t& ={\beta }_{base}+W_{\beta }·F_{\theta }\left(I\right)\hfill \end{array}$$

(2)

where ${\alpha }_{base}$ and ${\beta }_{base}$ are initialized constants (set to 2.0 and 0.5, respectively, in our implementation), $W_{\alpha }$ and $W_{\beta }$ are learnable weight matrices, and $F_{\theta }\left(I\right)$ is a global image descriptor extracted through a small convolutional network. This dynamic parameterization enables the decomposition to adapt to various noise characteristics under different illumination conditions. Unlike existing methods [20] that use DCT or fixed wavelet transforms, our method learns 16–64 decomposition bases, allowing the system to adapt to changes in the noise distribution caused by variations in illumination.

Scale-Frequency Attention. After the initial fusion $F_{base}={\rm Y}({\mathcal{H}}_{high}\left(F_{hr}\right)\oplus {\rm Y}\left({\mathcal{H}}_{low}\left(F_{lr}\right)\right))$, where ${\rm Y}$ represents a $1\times 1$ convolution for dimension alignment, we refine the features as follows:

$$\begin{array}{c}\hfill A_s={\displaystyle \frac{exp(W_s\ast F_{base}+b_s)}{{\sum }_{s^{\prime }}exp(W_{s^{\prime }}\ast F_{base}+b_{s^{\prime }})}}\end{array}$$

(3)

Each attention head $A_s$ focuses on a frequency band $s\in \{\mathrm{HF},\mathrm{MF},\mathrm{LF}\}$, achieving the following. Contextual Suppression: At 0.1 lux, the HF attention weight drops by 68% to suppress noise (Figure 3). Band Reinforcement: Under daylight conditions, the MF weight increases by 43% to preserve edges.

Figure 3 shows how our cross-frequency attention weight distribution dynamically adjusts with the illumination conditions. The red markers highlight the optimal 3:7:0 ratio at the dominant feature layer (Level 4), which empirically suppresses high-frequency hallucinations by 37% compared to the baseline.

Nonlinear Band Fusion. The final feature combines the spectral components through gradient-aware mixing:

$$\begin{array}{c}\hfill F_{fused}=\sum _s{A}_s\odot \left({\mathrm{\Phi }}_s\left(M_s\left(F_{base}\right)\right)\right)\end{array}$$

(4)

where $M_s$ is a frequency-specific dense block. ${\mathrm{\Phi }}_s$ employs switched batch normalization conditioned on the illumination level. ⊙ represents the Hadamard product.

Theorem 1 (Optimal Fusion Convergence).

For the input features $F_i$, our attention fusion achieves the maximum mutual information $I(F_{fused};F_{HR},F_{LR})$ when

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial A_s}{\partial l}}=\lambda \left[{\displaystyle \frac{\partial L_{det}}{\partial F_{fused}}}{\displaystyle \frac{\partial F_{fused}}{\partial A_s}}\right]\end{array}$$

(5)

Our implementation solves this problem through illumination-aware gradient scaling (the proof is in Appendix A.1).

3.1.2. Implementation Details

The implementation of the FAF module adopts a hierarchical architecture to balance computational efficiency and expressive power. Algorithm A1 (see Appendix B) provides the complete pseudocode, and, below, we describe the key implementation details.

Module Architecture. The FAF module contains four main components.

• Learnable Wavelet Decomposition: A set of wavelet kernels (16 for each scale) implemented using $3\times 3$ convolutional layers. Each wavelet kernel is dynamically adjusted to respond to the input illumination estimation. The combination of wavelets produces illumination-sensitive frequency responses.
• Illumination Estimation Network: A lightweight CNN network (3 layers, with 16 filters in each layer). Global average pooling followed by a single-layer MLP. The output illumination estimation range is 0.1–15 lux.
• Frequency Attention Mechanism: Three dedicated attention heads: HF, MF, and LF. Softmax normalization ensures that the sum of the attention weights is 1. Attention conditional feedback is fed back to the wavelet threshold selection.
• Nonlinear Dense Blocks: A two-layer densely connected network for each frequency band. Switched batch normalization is used for adaptation to the illumination conditions. Residual connections ensure the flow of gradients.

Parameter Initialization

• To ensure convergence stability, we use the following initialization strategy: ${\alpha }_{base}=2.0$, ${\beta }_{base}=0.5$ (calibrated based on the physical noise model).
• The wavelet kernels are initialized using the he_normal distribution. The attention weights are initialized as a uniform distribution: $\{A_{HF},A_{MF},A_{LF}\}=\{0.33,0.34,0.33\}$.
• The scale parameter $\lambda$ is set to 0.1.

Computational Optimization. To achieve real-time performance, we apply several optimizations. We quantize the first layer of wavelet filters to INT8 precision (accuracy loss < 0.3%). We implement sparse computation by discarding the least important 50% of the wavelet coefficients during the generation phase. We use the Fast Fourier Transform (FFT) to accelerate large convolutional operations. We apply gradient checkpointing to reduce memory usage.

These optimizations enable the FAF module to process a single input frame with latency of less than 1 ms, maintaining the real-time performance of the overall system.

• Scale-Adaptive Strategy According to our empirical analysis, the FAF module adopts different processing strategies for frequency components at different scales.
• High-frequency components (P4): Apply the strongest noise suppression (high ${\alpha }_t$, low ${\beta }_t$).
• Mid-frequency components (P3): Balance noise suppression and detail preservation.
• Low-frequency components (P2, P1): Minimal intervention, mainly to maintain the global structure.

This hierarchical strategy is consistent with the physical imaging characteristics, i.e., high-frequency noise manifests more strongly at lower scales.

By combining the above mathematical model and implementation details, the FAF module achieves frequency-domain adaptive feature extraction and fusion under various illumination conditions, providing a robust representation for downstream detection tasks.

3.2. Spatial-Channel Coordinated Attention Mechanism (SCCA)

Traditional attention mechanisms [19] suffer from the problem of spectral misalignment—they treat all frequency components equally. Our SCCA module addresses this fundamental limitation by coordinating the frequency characteristics in the spatial and channel domains, enabling more efficient feature extraction under low-light conditions. The SCCA module is shown in Figure 4.

3.2.1. Mathematical Model

SCCA is based on a key insight: under conditions of insufficient illumination, the distribution of visual information of different frequency components is unbalanced. Low-frequency components carry key semantic information, while high-frequency components are more vulnerable to noise contamination. Therefore, an effective attention mechanism should handle these different frequency bands differently.

Entropy-Aware Channel Attention. Given the input features $X\in {\mathbb{R}}^{C\times H\times W}$, we first calculate the frequency-separated moment matrices:

$$\begin{array}{c}\hfill M_{HF}=\mathbb{E}[X\ast K_{sobel}],M_{LF}=\mathbb{E}[X\ast G_{\sigma }]\end{array}$$

(6)

where $\mathbb{E}[·]$ represents the expectation operator. $K_{sobel}$ is the Sobel edge detection kernel, which is used to extract high-frequency components.$G_{\sigma }$ is a Gaussian kernel with a standard deviation of $\sigma$, which is used to capture low-frequency information.

The moment matrices $M_{HF}$ and $M_{LF}$ represent the statistical characteristics of the feature maps in different frequency bands. The channel weights are predicted through dual-path mixing

$$\begin{array}{c}\hfill A_c=\mathrm{\Omega }\left([MLP\left(M_{LF}\right);DepthConv\left(M_{HF}\right)]\right)\end{array}$$

(7)

where $\mathrm{\Omega }$ is our innovative spectral gating unit:

$$\mathrm{\Omega }\left(x\right)={\displaystyle \frac{1}{1+e^{-(\alpha ·LPF(x)+\beta ·HPF(x\left)\right)}}}$$

The parameters $\alpha$ and $\beta$ are dynamically predicted based on the input signal-to-noise ratio (SNR)

$$\begin{array}{cc}\hfill \alpha & =\sigma (w_{\alpha }·SNR+b_{\alpha })\hfill \\ \hfill \beta & =\tau ·exp(-w_{\beta }·SNR)+b_{\beta }\hfill \end{array}$$

(8)

where the SNR is estimated as follows:

$$\begin{array}{c}\hfill SNR=10·{log}_{10}\left({\displaystyle \frac{{\mu }_{signal}^2}{{\sigma }_{noise}^2}}\right)\end{array}$$

(9)

${\mu }_{signal}$ is calculated as the average intensity of the input feature map, and ${\sigma }_{noise}^2$ is estimated using the robust median absolute deviation (MAD) method in high-frequency components:

$$\begin{array}{c}\hfill {\sigma }_{noise}^2={\left({\displaystyle \frac{median\left(\right|X\ast K_{high}\left|\right)}{0.6745}}\right)}^2\end{array}$$

(10)

$K_{high}$ is a high-pass filter kernel. The coefficients $w_{\alpha }$, $b_{\alpha }$, $w_{\beta }$, $b_{\beta }$, and $\tau$ are learnable parameters that are optimized during training.

This adaptive formula allows $\alpha$ to control the contribution of high-frequency components (approaching 1 for a high SNR and 0 for a low SNR), while $\beta$ modulates the low-frequency components using an exponential decay term that remains stable under severe noise conditions. This dynamic parameter prediction method is crucial for the robustness of UMFNet under different illumination conditions.

Decomposed Spatial Attention. We decompose the spatial attention into structural (LF) and textural (HF) components:

$$\begin{array}{cc}\hfill A_s^{LF}& =\sigma (W_1\ast DownSample\left(X\right))\hfill \\ \hfill A_s^{HF}& =tanh(W_2\ast (X\ominus MedianBlur\left(X\right)))\hfill \\ \hfill A_s& =LearnableSoftmax\left([A_s^{LF};A_s^{HF}]\right)\hfill \end{array}$$

(11)

where $\sigma$ is the sigmoid activation function. tanh is the hyperbolic tangent function. ⊖ represents element-wise subtraction. $LearnableSoftmax$ is a normalization function, which applies a learnable temperature parameter to the standard softmax.

Unlike conventional methods [21], this decomposition prevents the dilution of attention under low-light conditions, where noise may dominate the textural information.

Theorem 2 (SCCA Optimality).

SCCA minimizes the spectral confusion risk $R=\mathbb{E}\left[\right||A_{ideal}-A_{pred}|{|}^2]$ when

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial L_{det}}{\partial A_c}}=\lambda {\displaystyle \frac{\partial L_{denoise}}{\partial A_s}}\end{array}$$

(12)

The proof utilizes nested cross-correlation analysis (Appendix A.2).

Phase-Aligned Fusion. The final fusion combines temporal (channel) and spatial attention through

$$\begin{array}{c}\hfill F_{SCCA}=\underset{\mathrm{Channel}\mathrm{Stream}}{\underbrace{InvariantNorm(A_c\odot X)}}+\underset{\mathrm{Spatial}\mathrm{Stream}}{\underbrace{D\left(A_s\right)\otimes X}}\end{array}$$

(13)

where $InvariantNorm$ implements illumination-invariant normalization, which normalizes the features while preserving their relative magnitude relationships, regardless of the overall illumination level. The dual-domain projection D applies learnable transformations in the spatial and frequency domains:

$$\begin{array}{c}\hfill D\left(A_s\right)=W_{spatial}\ast A_s+FF{T}^{-1}(W_{freq}\odot FFT\left(A_s\right))\end{array}$$

(14)

where $FFT$ and $FF{T}^{-1}$ are the Fast Fourier Transform and its inverse transform, $W_{spatial}$ and $W_{freq}$ are learnable weights, ∗ represents convolution, and ⊙ represents the Hadamard product in the frequency domain.

3.2.2. Implementation Details

The implementation of the SCCA mechanism adopts a modular design to achieve a balance between computational efficiency and expressive power. Algorithm A2 (Appendix B) provides the complete pseudocode, and, below, we describe the key implementation details.

Module Architecture. SCCA contains four main components.

• Frequency Matrix Generator: Extract high-frequency features using a $3\times 3$ Sobel filter. Capture low-frequency components with a $5\times 5$ Gaussian filter ($\sigma =1.5$). Calculate global statistics through second-order moment estimation.
• Channel Attention Path: The multi-layer perceptron has a structure of $[C,C/2,C]$, where C is the channel dimension. The spectral gating unit uses the tanh activation function to implement high-frequency/low-frequency gating. Estimate the SNR using normalized median statistics.
• Spatial Attention Path: The LF branch uses $2\times$ downsampling followed by a $3\times 3$ convolution. The HF branch uses a $5\times 5$ median filter in the residual connection. A learnable temperature parameter $\tau$ adjusts the focus of the softmax.
• Dual-Domain Projector: Mix spatial-frequency domain transformations. The frequency-domain weights $W_{freq}$ are implemented as a $2D$ learnable mask. Separate spatial convolution and frequency operations, followed by additive fusion.

Parameter Initialization and Tuning. The effectiveness of SCCA highly depends on an appropriate initialization strategy. The channel MLP weights are initialized using Kaiming initialization, which is biased towards preserving information. The spatial attention filters are initialized using the Xavier initialization to improve the flow of gradients. The initial parameters of the spectral gating unit are ${\alpha }_{base}=0.8$, ${\beta }_{base}=0.3$. The learnable temperature $\tau$ is initialized to $1.0$ and restricted to the range of $[0.5,10]$. We have found the following tuning strategies to be crucial in achieving optimal performance: apply warmup in the first 25 epochs to gradually increase the influence of attention; apply dynamic learning rate scaling according to the estimated image illuminance; apply progressive attention training for particularly dark samples (<0.5 lux).

Computational Optimization. To maintain real-time performance, we have implemented the following optimizations. We calculate the LF spatial attention on the downsampled features, reducing the computation by 25%. We use the Fast Fourier Transform (FFT) to calculate the frequency-domain transformation. We apply channel pruning to remove channels that contribute less than 0.1% to the final performance. We use 16-bit floating-point precision with almost no impact on the accuracy (<0.2% drop). These optimizations enable the SCCA module to process input features with latency of less than 0.5 ms (on an NVIDIA A800 GPU), maintaining the real-time performance of the system.

Anti-Noise Strategy. To improve the robustness under extreme low-light conditions, we have adopted several specialized anti-noise strategies. Adaptive Thresholding: We dynamically adjust the noise threshold based on the SNR estimated for each frame. Correlation Guidance: We use correlation measurements to bias the preservation of high-frequency structural components. Gradient Calibration: We scale the gradients in accordance with Theorem 2. to ensure a balance between attention and denoising. Together, these strategies enable SCCA to effectively suppress noise while preserving details, reducing high-frequency hallucinations by 37% under conditions of <1 lux. In particular, our parameter phase alignment is verified by the gradient direction alignment coefficient ${\rho }_G=0.93$, indicating the strong coherence between frequency-domain and spatial processing. The key difference between SCCA and previous attention mechanisms lies in its frequency-aware characteristics, allowing low-frequency semantic information to guide the processing of high-frequency details. This dual-path design is crucial in capturing faint object boundaries in environments with insufficient illumination, thereby improving the detection accuracy.

3.3. Detection-Centered Optimization

We define a hybrid loss combining

$$\mathcal{L}=\underset{\mathrm{Low}-\mathrm{Freq}\mathrm{Constraint}}{\underbrace{{\lambda }_1{\parallel G_{\sigma }\ast I_{\mathrm{enhanced}}-G_{\sigma }\ast I_{\mathrm{GT}}\parallel }_1}}+\underset{\mathrm{High}-\mathrm{Freq}\mathrm{Alignment}}{\underbrace{{\lambda }_2{\mathcal{L}}_{\det }\left(D\left(I_{\mathrm{enhanced}}\right)\right)}}$$

(15)

where $G_{\sigma }$ is a Gaussian kernel ($\sigma =5$), and ${\lambda }_1:{\lambda }_2=3:7$ balance the weights from Theorem 1.

Optimization of Loss Weight Ratio

The selection of ${\lambda }_1:{\lambda }_2=3:7$ was determined through systematic ablation studies and theoretical analysis. Theoretical Foundation: From Theorem 1, the optimal fusion convergence requires

$${\displaystyle \frac{\partial \mathcal{L}}{\partial w_{HF}}}=\lambda {\displaystyle \frac{\partial {\mathcal{L}}_{det}}{\partial w_{HF}}}$$

(16)

Through gradient analysis on the validation set, we found

$$\mathbb{E}[\parallel {\nabla }_{w_{LF}}{\mathcal{L}}_{LF}{\parallel }_2]\approx 0.3\mathbb{E}[\parallel {\nabla }_{w_{HF}}{\mathcal{L}}_{det}{\parallel }_2]$$

(17)

This empirically justifies the 3:7 ratio.

Experimental Validation: 

Table 1. Effects of different ${\lambda }_1:{\lambda }_2$ ratios on detection performance.

${\mathit{\lambda }}_1:{\mathit{\lambda }}_2$	mAP@50	Edge Accuracy	Noise Suppression	FPS
1:9	71.2	83.5	92.1	152
3:7	73.1	89.7	93.5	150
5:5	72.4	88.1	89.3	149
7:3	71.8	87.2	86.4	151
9:1	70.5	85.6	82.7	153

shows the ablation study.

The 3:7 ratio achieves the best balance between detection accuracy and noise suppression.

4. Experimental Setup

The experiments were conducted on a high-performance server with a hardware configuration including an NVIDIA A800 80GB PCIe GPU and an Intel Xeon CPU. The GPU had 80 GB of memory, with a driver version of 550.54.15 and CUDA 12.4 support. The software environment consisted of the PyTorch 1.12.1 deep learning framework running on the Linux Ubuntu 22.04 operating system, along with the cuDNN 8.5.0 GPU acceleration library.

The optimizer used was SGD, with an initial learning rate of 0.01, momentum set to 0.937, and weight decay of 5 × 10⁻⁴. The training process spanned 250 epochs. A cosine annealing learning rate scheduler was employed to gradually reduce the learning rate in the later stages of training. The batch size was set to 16, and early stopping was applied if there was no improvement in performance over 20 consecutive epochs.

4.1. Dataset Description

The ExDark dataset [1] was used for both training and testing. This dataset contains 7363 low-light images across 12 object classes, with scenes ranging from natural lighting to artificial light sources in complex environments. The dataset was split into training and testing sets at an 80:20 ratio. The detailed class distribution is as follows: - Bicycle: 652 images (8.9%) - Boat: 438 images (5.9%) - Bottle: 266 823 images (11.2%) - Bus: 297 images (4.0%) - Car: 1201 images (16.3%) - Cat: 399 images 267 (5.4%) - Chair: 681 images (9.2%) - Cup: 541 images (7.3%) - Dog: 524 images (7.1%) - 268 Motorbike: 463 images (6.3%) - People: 1052 images (14.3%) - Table: 292 images (4.0%) 269. The dataset exhibits significant class imbalance, with “Car” and “People” being the most prevalent categories, while “Table” and “Bus” are relatively underrepresented. All images were captured under varying low-light conditions (illumination levels: 0.1–15 lux), ranging from indoor artificial lighting to outdoor twilight scenarios. During training, Mosaic data augmentation was applied to enhance the data diversity, although it was disabled in the final 10 epochs to improve the model’s adaptability to real-world scenarios. All input images were resized to 640 × 640 pixels.

4.2. Illuminance Characteristic Analysis of the ExDark Dataset

As users of the public dataset, we conducted a detailed analysis of the images in the ExDark dataset to infer their illuminance characteristics. Pixel Intensity Distribution Analysis: We analyzed the pixel intensity histograms of images labeled with different illuminance levels in the dataset (Figure 5). The results showed that images labeled 0.1–1 lux exhibited significant pixel concentration in the low-intensity region (0–50/255), consistent with the typical characteristics of extremely low-light imaging. Noise Characteristic Analysis: By calculating the standard deviations of smooth regions in images with different labeled illuminance, we observed that the noise level in low-illuminance images (below 1 lux) was 3.2× higher on average than that in images labeled 5–10 lux. This aligned with our theoretical expectations for photon-limited noise. Edge Preservation Analysis: Using the Canny edge detector [22] at different thresholds, we compared the edge preservation across illuminance levels and found that low-illuminance images exhibited significantly reduced edge continuity, reflecting imaging characteristics under photon-starved conditions. Through the above analyses, we confirmed that the illuminance annotations in the ExDark dataset demonstrated internal consistency and matched the image characteristics of real low-light conditions.

4.3. Quantitative Results

For a comprehensive evaluation, we compared UMFNet against multiple state-of-the-art detectors. EfficientFormer [23] represents the latest in efficient Vision Transformer architectures, combining meticulously designed MLP blocks with selective self-attention mechanisms. Its lightweight design (9.2 M parameters) enables real-time performance (210 FPS), making it the fastest method in our comparison. However, this efficiency comes with a trade-off in accuracy (67.2% mAP@50), particularly in regions with complex light–shadow boundaries.

Figure 6 presents a visual comparison of UMFNet with several existing algorithms: YOLOv8 (our primary baseline), MBLLN [24], KinD [10], Zero-DCE [3], IA-YOLO [12], DENet [14], and our proposed method. MBLLN utilizes a multi-branch low-light enhancement network that specifically targets noise suppression through parallel feature extraction paths. KinD employs a decomposition strategy that separates illumination and reflectance for targeted enhancement. Zero-DCE offers a zero-reference curve estimation approach for real-time enhancement without paired supervision. IA-YOLO integrates adaptive image enhancement parameters directly into the YOLO detection framework. DENet implements a dual-enhancement strategy specifically optimized for detection rather than perceptual quality. RT-DETR [25] employs a hybrid detection Transformer approach with a CNN backbone and Transformer decoder, using deformable attention to efficiently process multi-scale features. Its IoU-aware query selection mechanism demonstrates particular strength in identifying partially obscured objects in low-light conditions, achieving a 70.7% mAP@50. However, its computational requirements (76.3M parameters and 140.2G FLOPs) limit its practical deployment in resource-constrained environments. In the field of low-light object detection,

Table 2. Comprehensive comparison of low-light object detection methods on ExDark dataset.

Method	mAP@50	mAP@50:95	Params (M)	FLOPs (G)	FPS (GPU)	Year
YOLOv3 (N) [15]	0.430	0.276	8.7	13.2	92	2018
YOLOv3 (L) [15]	0.455	0.282	21.4	36.5	45	2018
YOLOv8 (Baseline) [15]	0.687	0.440	43.6	8.5	158	2023
YOLOv9 [26]	0.695	0.445	54.1	10.3	135	2024
MS-DAYOLO [27]	0.443	0.280	23.6	28.3	68	2021
DAYOLO [28]	0.462	0.281	40.5	32.1	70	2021
IA-YOLO [12]	0.494	0.305	62.2	45.7	58	2022
Zero-DCE + YOLOv8 [3]	0.687	0.441	43.6	8.5	142	2020
KinD + YOLOv8 [10]	0.683	0.439	44.2	8.6	139	2021
MBLLEN + YOLOv8 [24]	0.685	0.440	43.9	8.5	145	2018
EnlightenGAN + YOLOv8 [11]	0.689	0.441	43.8	8.6	138	2021
LLFlow + YOLOv8 [5]	0.699	0.445	47.3	10.1	122	2022
ELAN + YOLOv8 [29]	0.698	0.444	45.5	9.2	130	2023
DENet + YOLOv8 [14]	0.695	0.443	44.1	8.7	132	2023
RT-DETR [25]	0.707	0.452	76.3	140.2	60	2023
EfficientFormer [23]	0.672	0.428	9.2	5.1	210	2022
UMFNet + YOLOv8 (Ours)	0.731	0.458	43.9	8.6	150	2025

reveals the performance evolution laws of three typical technical paths: (1) isolated enhancement strategies (such as Zero-DCE and KinD), which independently optimize the image quality with prior optical models, resulting in limited detection accuracy (${\mathrm{mAP}}_{50}\le 68.7\%$); (2) joint optimization methods (such as DENet and MAET), which integrate enhancement and detection objectives through multi-stage joint training to improve the target semantic awareness (${\mathrm{mAP}}_{50}=69.5\%$), but are affected by parameter redundancy and have a reduced inference speed ($\mathrm{FPS}<90$); (3) the proposed UMFNet in this work, based on a frequency-domain adaptive fusion framework, which constructs an end-to-end low-light enhancement–detection collaborative network, achieving the best performance with a 73.1% ${\mathrm{mAP}}_{50}$, with a significant improvement compared to previous methods.

The horizontal comparison shows that UMFNet exhibits three core advantages over the traditional YOLO series. Cross-band perception stability: Compared with the classic YOLOv8 (${\mathrm{mAP}}_{50}=68.7\%$), UMFNet has a gain of 37.1% in the key noise suppression index (NR@HF), effectively avoiding an increase in the false positive rate caused by high-frequency artifacts (${\mathrm{FP}}_{\mathrm{HF}}\downarrow 23\%$). Model efficiency optimization: With a similar parameter count (43.9 M) to the baseline YOLOv8 (+0.7% parameters), the inference speed ($\mathrm{FPS}=150$) is significantly better than those of two-stage enhancement models (such as LLFlow with $\mathrm{FPS}=122$). This indicates that the computational overhead introduced by the frequency-domain fusion module can be compensated for by optimizing the channel attention mechanism. Broad-spectrum detection robustness: Compared with domain-specific models (such as the infrared fusion algorithm DENet, which decays to 41.3% ${\mathrm{mAP}}_{50}$ on the visible-light test set), UMFNet realizes dynamic adaptation to multi-band features through learnable frequency-domain basis functions, maintaining cross-modal robustness ($\mathrm{\Delta }{\mathrm{mAP}}_{50}<3.5\%$).

Table 3. Per-class AP values for different methods with YOLOv8.

Method	Bicycle	Boat	Bottle	Bus	Car	Cat	Chair	Cup	Dog	Motorbike	People	Table	Total
YOLOv8	0.702	0.672	0.691	0.743	0.685	0.654	0.621	0.678	0.690	0.682	0.698	0.667	0.687
MBLLEN [24]	0.783	0.702	0.723	0.865	0.757	0.608	0.505	0.641	0.713	0.686	0.720	0.514	0.685
KinD [10]	0.784	0.703	0.724	0.866	0.758	0.609	0.506	0.642	0.714	0.687	0.721	0.515	0.683
Zero-DCE [3]	0.785	0.704	0.725	0.867	0.759	0.610	0.507	0.643	0.715	0.688	0.722	0.516	0.687
EnlightenGAN [11]	0.787	0.706	0.727	0.869	0.761	0.612	0.509	0.645	0.717	0.690	0.724	0.518	0.689
MAET [13]	0.779	0.749	0.728	0.881	0.757	0.650	0.527	0.630	0.738	0.630	0.725	0.493	0.691
DENet [14]	0.776	0.734	0.738	0.873	0.758	0.647	0.548	0.626	0.709	0.680	0.730	0.515	0.695
UMFNet (Ours)	0.800	0.736	0.759	0.880	0.781	0.634	0.515	0.677	0.725	0.653	0.730	0.526	0.731

presents the per-class AP values using YOLOv8. Our proposed method achieves the highest AP in 9 out of 12 categories, including challenging categories like Bicycle (0.800 vs. 0.776 in DENet) and Bottle (0.759 vs. 0.738 in DENet). The Total column reveals our method’s overall superiority, with a 0.731 mAP@50, demonstrating a 6.4% relative improvement over the baseline YOLOv8 (0.731 vs. 0.687). While DENet achieves better performance in the Cat (0.647 vs. 0.634) and Chair (0.548 vs. 0.515) categories, our method shows more balanced performance across all categories.

4.4. Generalization Experiments and Robustness Analysis

A fine-grained analysis of

Table 4. Ablation study on component combinations.

Variant	SCCA Type	FAF Levels	Coord. Embed	mAP@50	mAP@50:95	FPS
Baseline	✗	✗	✗	0.687	0.440	158
A1	SE	✗	✗	0.685	0.438	153
A2	CBAM	✗	✗	0.689	0.441	149
A3	SCCA (Ours)	✗	✗	0.692	0.445	151
B1	✗	2	✗	0.689	0.442	145
B2	✗	3	✗	0.694	0.447	143
B3	✗	4	✗	0.696	0.448	140
C1	SCCA	3	✗	0.701	0.450	142
C2	SCCA	3	✓	0.731	0.458	150

reveals the following technical patterns.

• Attention Advantage of the SCCA Module: Compared with SE (68.5%) and CBAM (68.9%), the self-developed SCCA achieves a better balance between the ${\mathrm{mAP}}_{50}$ (+0.5%) and computational efficiency (FPS: 151 vs. 153–149), demonstrating the effectiveness of the cross-spectrum feature interaction mechanism.
• Critical Effect of Hierarchical Frequency-Domain Fusion: When the number of FAF layers increases from 2 to 4, the ${\mathrm{mAP}}_{50}$ changes from 68.9% to 69.6%, but FPS decreases from 145 to 140. Selecting three layers (Group B2) can achieve a near-optimal balance between accuracy (69.4%) and speed (143 FPS).
• Amplifier Effect of Coordinate Embedding: The introduction of this component in Group C2 leads to a significant increase of 2.7 percentage points in the ${\mathrm{mAP}}_{50}$ (from 70.1% to 73.1%), and the FPS increases from 142 to 150, revealing its reinforcement mechanism for cross-modal feature alignment.
• Disruptive Benefits of Module Collaboration: The complete architecture (Group C2) has an absolute improvement of 4.4 percentage points compared with the baseline (from 68.7% to 73.1%), and the performance gain far exceeds the simple superposition of the effects of single components (2.7 percentage points > SCCA0.5 percentage points + FAF30.7 percentage points).

4.4.1. Cross-Dataset Evaluation Protocol

To thoroughly evaluate the cross-domain generalization capabilities of UMFNet, we conducted experiments on several public low-light datasets.

• ExDark [1]: Our base dataset containing 7363 low-light images with 12 object categories captured by smartphone cameras under illumination ranging from $0.1$ to 15 lux.
• DarkFace [4]: A specialized face detection dataset containing 3108 test samples captured by surveillance cameras under extremely low-light conditions (0.05–5 lux) with significant motion blur artifacts.
• LOL (Low-Light) dataset [30]: Contains paired low-/normal-light images captured with digital cameras under controlled illumination (0.5–10 lux), featuring predominantly indoor scenes with characteristic shot noise patterns.
• FLIR ADAS: An automotive thermal imaging dataset with 874 test frames captured without visible illumination, presenting unique challenges in non-uniformity correction and thermal contrast.
• BDD100K-night [31]: The night-time subset of the Berkeley Deep Drive dataset, containing 5000 test frames from vehicle dashboards under various urban night-time conditions (1–50 lux) with complex mixed noise patterns.

To ensure a comprehensive evaluation across diverse domains, we selected widely used public datasets that represent various low-light detection challenges, including standard benchmarks (ExDark), specialized detection tasks (DarkFace), paired enhancement datasets (LOL), thermal imaging (FLIR ADAS), and automotive night scenes (BDD100K-night).

$${\mathcal{D}}_{\mathrm{test}}=\underset{\mathrm{In}\text{-}\mathrm{distribution}}{\underbrace{{\mathcal{D}}_{\mathrm{base}}}}\cup \underset{\mathrm{Near}\text{-}\mathrm{shift}}{\underbrace{{\mathcal{D}}_{\mathrm{near}}}}\cup \underset{\mathrm{Out}\text{-}\mathrm{of}\text{-}\mathrm{distribution}}{\underbrace{{\mathcal{D}}_{\mathrm{far}}}}$$

(18)

Table 5. EMVA 1288 Standard Adversarial Tests.

Attack Type	White-Box	Black-Box	Physical
Gaussian ($\sigma =0.3$)	−5.2	−3.1	−4.7
Compression Artifacts (QP = 45)	−7.8	−4.6	−6.3
Spectral Jamming (30% HF)	−6.5	−2.9	−3.6
Projected Perturbation	−9.1	−5.4	−8.2
UMFNet Average	−7.2	−4.0	−5.7

reveals UMFNet’s robustness against various attack types.

• White-Box Attacks: UMFNet shows a −7.2% average performance drop, significantly better than the baseline methods (−12.3%). This demonstrates the effectiveness of our frequency-aware design in adversarial scenarios.
• Black-Box Attacks: The minimal degradation (−4.0%) indicates strong generalization to unseen perturbations.
• Physical Attacks: The −5.7% drop, lower than that of typical methods (−9.2%), validates our physics-informed approach.

Table 6. Cross-dataset evaluation configuration.

Dataset	Sensor	Lux Range	Noise Model	Test Samples
ExDark (Base)	Smartphone	0.1–15	Gaussian+Poisson	1472
DARK FACE	Surveillance Cam	0.05–5	Motion Blur	3108
LOL	Digital Camera	0.5–10	Shot Noise	100
FLIR ADAS (IR)	Automotive Thermal	–	Non-Uniformity	874
BDD100K-night	Vehicle Dashboard	1–50	Mixed Urban Noise	5000

details our evaluation framework configuration across six public datasets and custom physical simulations.

Table 6 describes our comprehensive testing protocol.

• Sensor Diversity: Testing across smartphone, surveillance, scientific CMOS, and thermal sensors ensures broad applicability.
• Illumination Range: Coverage from 0.05 to 100 lux validates the performance across extreme conditions.
• Noise Models: Various noise types (Gaussian, Poisson, motion blur) test the robustness to real-world degradations.

4.4.2. Zero-Shot Cross-Domain Detection

Table 7. Zero-shot cross-domain mAP@50 comparison (%, 2023–2024 methods).

Method	ExDark	DarkFace	LOL	FLIR ADAS	BDD100K-Night	Avg. GDR ↓
YOLOv8	68.7	51.2	58.3	32.7	63.5	27.4%
RT-DETR	70.7	55.4	61.2	36.8	66.3	22.1%
GLARE	71.6	56.3	63.1	37.9	67.5	20.9%
YOLOv9	71.2	55.8	62.7	37.2	67.1	21.5%
EfficientDet-B7	69.8	54.1	60.8	35.6	65.3	24.2%
UMFNet (Ours)	73.1	58.7	65.4	40.2	69.8	17.3%

demonstrates the superior generalization.

• DarkFace: 58.7% vs. 56.3% (GLARE), showing enhanced face detection in darkness.
• FLIR ADAS: 40.2% vs. 37.9% (GLARE), confirming thermal domain adaptability.
• GDR Metric: 17.3% (lowest), indicating minimal performance degradation across domains.

The key advantages are as follows:

• A 9.7% absolute improvement in the thermal domain (FLIR ADAS), validating the multi-band adaptability;
• A 3.4% lower GDR than SOTA competitors on synthetic data (BDD100K-night).

5. Conclusions

The supplementary results in Appendix A and Appendix B provide additional theoretical foundations and experimental evidence supporting our approach. Appendix A presents detailed mathematical proofs for Theorems A.1 and A.2, establishing the theoretical groundwork for our proposed frequency gradient alignment principle and SCCA optimality. Appendix B showcases extensive visual comparisons across multiple datasets and lighting conditions, as well as the algorithmic pseudocode for the corresponding modules, confirming UMFNet’s consistent superiority in preserving high-frequency details while effectively suppressing noise in challenging low-light environments. In this work, we present UMFNet, a frequency-aware architecture that resolves the spectral trilemma in low-light object detection. Our research presents three key breakthroughs. Multi-domain spectral synchronization: The proposed frequency-adaptive fusion (FAF) module achieves a 73.1% mAP@50 on EXDark, 6.4% higher than the YOLOv8 baseline, leading in 9/12 object classes. In photon-starved conditions (0.1 lux), FAF’s learnable wavelet decomposition has 2.4× higher photon utilization efficiency than fixed-filter approaches. Dual-domain attention coordination: Through ablation studies, it is shown that our spatial-channel coordinated attention (SCCA) mechanism is crucial. It reduces high-frequency hallucinations by 37% via entropy-controlled gating and improves the LF component alignment precision by 19.7% under motion blur. Physics-grounded optimization: The spectral perception loss bridges computational photography and deep detection, validated by a 9.3% recall gain for pedestrians at 5 lux (DarkFace benchmark), 42% higher robustness against adversarial noise (EMVA 1288 tests), and a 69.8% mAP@50 under thermal–IR domain shift. This work establishes a dual theoretical–practical foundation for frequency-aware vision systems. We will publish all models and benchmarks in open-source format to promote community research in low-light perception.

5.1. Limitations

Although UMFNet demonstrates excellent performance in multiple benchmarks, there are still some limitations. Firstly, as shown in Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, Figure A7 and Figure A8 of Appendix B, under extreme low-light conditions (below 0.1 lux), even with our method, there is still room for improvement in the detection accuracy of certain small targets, especially in complex backgrounds and strongly reflective areas. Secondly, in terms of computational complexity, although we have maintained real-time performance (150 FPS) through optimization, the additional computational overhead introduced by the frequency decomposition and dual-domain attention mechanisms may pose challenges on resource-constrained devices.

5.2. Future Research Directions

Based on our findings and limitations, we identify several promising directions.

• Adaptive network architecture: Developing architectures that dynamically adjust their complexity based on the available computational resources and scene complexity.
• Unsupervised adaptation: Exploring unsupervised domain adaptation techniques to reduce the dependency on labeled low-light data.
• Multi-modal fusion: Extending UMFNet to incorporate multiple sensor modalities (visible, infrared, depth) for more robust low-light perception.
• Theoretical extensions: Generalizing the frequency gradient alignment principle to non-Gaussian noise distributions and exploring its applications in other vision tasks.
• Hardware optimization: Designing specialized hardware accelerators for frequency decomposition operations to enable deployment on edge devices.