SemaFire-YOLO: A Lightweight and Robust Fire-Smoke Detection Model via Semantic Enhancement and Frequency-Aware Perception

Highlights

What are the main findings?

• First, a lightweight fire and smoke detection model (SemaFire-YOLO) integrating semantic enhancement and frequency domain perception modeling is proposed to solve the drawbacks of traditional fire detection methods.
• Second, the model is optimized with four key modules (LLaVA semantic guidance, Haar wavelet-based downsampling, convolution modulation mechanism, Dynamic Tanh normalization) to enhance target representation, retain high-frequency features and improve modeling efficiency.
• Third, experimental results show SemaFire-YOLO achieves an mAP@0.5 of 64.30%, outperforming mainstream lightweight YOLO models in fire detection accuracy.

What are the implications of the main findings?

• Theoretically, it enriches the application of computer vision in emergency rescue and provides a reference paradigm for optimizing similar special target detection algorithms.
• Practically, it improves the accuracy and timeliness of early fire detection, solves practical pain points of traditional methods, and supports the intelligent upgrade of fire emergency monitoring and rescue.

Abstract

Accurate detection in the early stages of a fire is a crucial prerequisite for the efficient implementation of fire suppression and emergency rescue operations. Its accuracy and timeliness directly affect the control of disaster loss severity. Traditional fire detection methods mainly include three categories, which are manual inspection, sensor detection, and visual recognition. However, manual inspection is restricted by labor costs and time efficiency, making it difficult to achieve large-scale, high-frequency and real-time fire monitoring. Sensor detection is easily interfered by environmental factors such as temperature, humidity, and dust, leading to frequent false alarms and missed alarms. Visual recognition technology has shortcomings in aspects such as detailed feature perception, dynamic scene modeling, and reasoning robustness in complex environments, making it difficult to meet the requirements of high-precision detection. To address these issues, this study innovatively proposes a lightweight fire and smoke detection model based on semantic enhancement and frequency domain perception modeling, which is named the SemaFire you only look once (SemaFire-YOLO) model. The model constructs a large language and vision assistant (LLaVA) semantic guidance module, which uses a large language model to understand and guide the semantic features of images, thereby enhancing the saliency representation intensity of small and weak target regions. Then, a Haar wavelet-based downsampling module is adopted, which compresses spatial information while preserving high-frequency features such as flame edges and smoke textures, improving the accuracy of target recognition. Next, the convolution modulation mechanism is introduced to replace the traditional attention mechanism, enhancing the overall modeling efficiency and reducing computational overhead. Finally, a Dynamic Tanh normalization module is adopted to replace the batch normalization module in the traditional YOLO algorithm, strengthening the model’s representation stability and reasoning robustness under unstable input distributions. Experimental results show that the SemaFire-YOLO model achieves a mean average precision (mAP@0.5) of 64.30% on the fire image dataset, which is 0.8, 2.0, 0.6, and 3.8 percentage points higher than that of mainstream models such as YOLOv5n, YOLOv8n, YOLOv11n, and YOLOv12n, respectively. It exhibits better boundary detection capability and practical deployment potential. Through visual analysis, the results indicate that the improved SemaFire-YOLO model achieves more accurate detection and higher confidence in actual complex scenarios, further verifying the model’s robustness and accuracy in complex scenarios such as low contrast and dynamic fire conditions.

1. Introduction

As a high-frequency and extremely destructive sudden disaster, fires occur extensively in various locations such as residential areas, industrial plants, and forests, posing a severe threat to public safety and social stability. Therefore, constructing an early fire detection and early warning system with high sensitivity and accuracy is of great significance for reducing casualties and property losses. At present, fire detection technologies mainly fall into three categories, which are manual inspection, sensor detection, and visual recognition. The traditional manual inspection method is inefficient and susceptible to subjective factors, with potential risks such as operational fatigue and observation omissions. Sensor detection methods, such as smoke sensors, thermal sensitive elements, and gas sensors, mainly trigger alarms by detecting smoke concentration, temperature changes, or harmful gas content [1,2,3,4,5]. These methods have a simple structure and flexible deployment. However, limited by their working mechanism, they often fail to respond promptly in the early stages of a fire and are prone to false alarms and missed alarms in complex environments. In addition, sensor systems usually cannot accurately locate the fire source and require regular maintenance and replacement, therefore the costs of system deployment, operation, and maintenance are relatively high. Visual fire detection based on image processing has attracted widespread attention due to its advantages of low cost, flexible deployment, and rapid response. Early image detection methods usually relied on manual extraction of image features (e.g., color [6], texture [7], and geometric shape [8]) and combined them with traditional machine learning algorithms to complete flame region extraction and classification. A multi-stage fire detection method was proposed that integrates color characteristics, shape transformation, and optical flow information, improving the accuracy, real-time performance, and anti-interference ability of computer vision technology in early fire detection [9]. However, in real-world scenarios, these methods often exhibit poor generalization ability and robustness due to the complexity of fire morphology and diverse environmental interferences, making it difficult to meet the needs of practical applications.

With the rapid development of deep learning technology and the substantial improvement in graphics processing unit (GPU) computing capabilities, visual perception methods based on deep neural networks (DNNs) have gradually emerged as the mainstream research direction in fire detection. A large number of studies has demonstrated that DNN models exhibit superior feature extraction capabilities and expressive generalization performance in flame and smoke detection tasks, while achieving higher detection accuracy and response efficiency [10]. Meanwhile, DNN models possess enhanced adaptability to complex backgrounds, target occlusions, and multi-scale features, rendering them more suitable for large-scale monitoring scenarios characterized by complex spatial structures and variable environmental conditions [11].

Currently, fire detection methods based on deep learning are mainly divided into two-stage methods and one-stage methods. Common two-stage methods include faster region-convolutional neural network (R-CNN) [12], libra R-CNN [13], and region-based fully convolutional networks (R-FCN) [14], whose main advantage lies in high detection accuracy. Common one-stage methods include RetinaNet [15], you only look once (YOLO) algorithms [16,17], and EfficientDet [18]. These methods perform better in real-time detection and are widely used in lightweight detection. Based on these basic networks, researchers have successively proposed a variety of improved methods for flame and smoke detection, achieving remarkable results. For example, an attention mechanism was introduced into YOLOv5 to enhance feature expression capabilities, which significantly improved fire detection accuracy [19]. However, this method still showed a high missed detection rate for weak flames and thin smoke in the early stages of fires. A fire detection system was built for smart cities based on YOLOv8, which demonstrated good robustness in multi-scale and multi-directional scenarios [20]. To achieve more lightweight deployment, an ultra-lightweight fire detection network was proposed to run on embedded devices, ensuring real-time performance while taking into account detection accuracy [21]. To address the issues of occlusion and dense smoke interference, Ren et al. [22] developed FCLGYOLO, which improves the fire source localization capability under complex conditions through an enhanced feature fusion strategy.

In addition, to further address the issues of high false alarm rates and insufficient detection accuracy, researchers have also conducted multi-faceted explorations on attention mechanisms and feature fusion structures. Aiming at the difficulty of small target recognition, the simple attention module (SimAM) 3D attention mechanism and four-scale detection architecture were introduced into YOLOv5, which significantly enhanced the perception ability of fine-grained fire and smoke targets [23]. A dynamic attention mechanism was added to the YOLOv8 framework, effectively reducing the false alarm probability in smoke detection [24]. However, there is still a high risk of false alarms under extreme weather conditions. To solve this problem, hybrid attention mechanisms were combined with a multi-scale convergent coordinated pyramid structure, improving the detection rate of smoke targets at different scales [25]. An adjustable dynamic convolution module was integrated into the YOLOv5 backbone network and combined with the shuffle attention mechanism [26]. Through this integration and combination, the model’s focusing ability on regions of interest was optimized, its detection performance was improved, and at the same time, its computational complexity was reduced. A self-attention smoke detection network was proposed and combined with a feature fusion module to alleviate the false positive problem in complex backgrounds [27]. To enhance multi-scale feature fusion, a small-target detection branch and an attention module were incorporated into the YOLOv5 architecture, and the spatial pyramid pooling-fast (SPPF) and path aggregation network (PANet) structures were adjusted accordingly [28]. Although a certain improvement in detection accuracy was achieved, the model complexity and computational overhead also increased accordingly, affecting the efficiency of real-time deployment.

Deep learning-based fire detection methods have achieved significant progress in terms of detection accuracy and generalization ability, yet some limitations still exist. Traditional image detection models rely solely on visual features for target recognition and lack in-depth understanding of image semantics. Particularly in the context of complex scenarios, weak fire conditions, and intense background interference, such methods often suffer from issues including blurred target boundaries, scattered response signals, and elevated false detection rates. At the same time, traditional downsampling operations, such as strided convolution and pooling, often lead to severe loss of high-frequency detailed features such as smoke edges and flame textures while compressing spatial information. Feature modeling methods based on self-attention mechanisms have high computational costs and limited ability to model aperiodic and dynamically diffused features, making it difficult to balance efficiency and expressive ability. In addition, common normalization methods, such as batch normalization (BN) and layer normalization (LN), have insufficient adaptability to small-batch inputs or dynamic environments, resulting in unstable performance during the inference phase.

To address the aforementioned issues, this study proposes a fire detection model termed SemaFire you only look once (SemaFire-YOLO), which integrates semantic awareness, frequency-domain enhancement, lightweight spatial modeling, and normalization adaptability capabilities. Its core innovations are embodied in the following four aspects.

• To the best of our knowledge, this work is the first to introduce the multimodal large language and vision assistant (LLaVA) auxiliary mechanism into fire smoke detection tasks. The images understanding capability of LLaVA is introduced into fire detection tasks. It automatically assesses image quality, mitigates blurring, low contrast, and insufficient illumination, guides denoising, contrast enhancement, and edge refinement, and substantially improves the saliency and detectability of target regions.
• A frequency-domain aware downsampling structure is constructed based on two-dimensional (2D) Haar wavelet transform. It decomposes the input feature map into low-frequency and multi-directional high-frequency sub-bands. It compresses spatial dimensions while preserving local details such as edges and textures, thereby significantly enhancing structural perception of flame boundaries and smoke diffusion regions and improving small-target detectability.
• A lightweight convolutional modulation modeling mechanism (ConvMod) is employed to replace the traditional self-attention module. This mechanism introduces a combination of large-kernel depth wise convolution and Hadamard product. It establishes long-range contextual modeling capabilities, while circumventing inference latency induced by high-order matrix operations. This not only substantially reduces computational complexity but also enhances the model’s real-time inference performance and the accuracy of dynamic feature capture in high-resolution scenarios.
• A Dynamic Tanh (DyT) normalization module is integrated to implement the statistics-free normalization alternative strategy. Without relying on the computation of mean and variance, the module achieves dynamic range compression via learnable scaling and affine parameters. This approach effectively mitigates the performance degradation of traditional normalization methods in small-batch or unstructured inference scenarios, while enhancing the model’s stability and adaptability.

In summary, the proposed structural innovations—encompassing frequency-domain enhancement, spatial modeling, and normalization mechanisms—are all developed to address the core challenges in fire and smoke detection. The constructed model demonstrates superior boundary preservation capability, dynamic feature modeling capability, and structural stability under complex backgrounds, thereby providing effective technical support for the development of a high-precision and low-latency fire visual perception system.

2. Model Design

2.1. The Architecture of the SemaFire-YOLO Model

To achieve efficient detection and intelligent perception of flame and smoke targets, this study proposes an improved detection framework based on YOLOv12, which integrates frequency-domain enhancement, normalization optimization, lightweight modeling, and semantic guidance mechanisms. The overall structure of the framework is illustrated in Figure 1, which consists of four core components: the image semantic enhancement module (incorporating large language model-assisted processing), backbone network (Backbone), feature fusion module (Neck), and detection head (Head). The image semantic enhancement module constitutes a key innovation in this study. Based on the multimodal large language model LLaVA, this module conducts semantic understanding of input images. By generating natural language descriptions for the images, it enables “language-guided image enhancement”. On this basis, it integrates image processing techniques to perform targeted enhancement of image structures, which significantly improves the saliency and feature separability of fire regions and provides clearer, more distinguishable input information for the subsequent YOLO detection module.

The area-attention enhanced cross-feature (A2C2f) module employed in the original YOLOv12 exhibits a certain level of cross-channel and cross-scale modeling capabilities. However, due to its dependence on the multi-head self-attention mechanism, it imposes high inference costs and demonstrates low sensitivity to dynamic features (e.g., flame flickering and smoke drifting) [29]. To address this issue, this study proposes the A2C2f_ConvMod module. By introducing large-kernel depthwise convolution and a Hadamard modulation mechanism to replace the self-attention path, the module expands the receptive field, effectively reduces computational complexity, and enhances the modeling capability for local edges and spatial structures. To mitigate edge blurring and target loss caused by traditional downsampling operations, this study introduces a HWD structure into the multi-scale path of the Neck module. By extracting high-frequency textures and directional features from images via frequency-domain decomposition, this structure not only preserves spatial compression efficiency but also significantly enhances the model’s perception capability for tiny and blurry targets. It is particularly suitable for fire scenarios with low contrast. In addition, to address the reliance of deep feature normalization on batch distribution, this study introduces the DyT normalization mechanism based on the cross-stage partial with kernel size 2 (C3K2) module. This mechanism constructs a normalization mapping via a bounded activation function and learnable scale parameters. It eliminates dependence on batch statistics and enhances the model’s expressive stability and inference robustness under small-sample, high-noise, and unstructured input conditions.

In conclusion, the proposed SemaFire-YOLO model constructs a unified detection framework integrating semantic-guided enhancement, frequency-domain modeling, lightweight structural perception, and dynamic normalization. Without significantly increasing computational complexity, it achieves superior perceptual capability, structural preservation capability, and semantic consistency, thereby providing an effective solution for high-precision and deployable fire detection.

2.2. LLaVA

To further enhance the modeling capability of fire detection models for flame and smoke targets in complex scenarios, this study integrates the multimodal semantic understanding module LLaVA into the SemaFire-YOLO framework, and constructs a language-guided image structure enhancement mechanism. This module serves as the model’s pre-perception component. It utilizes the large language model to conduct semantic analysis on input images and combines image processing tools to complete regional enhancement. The module aims to improve input quality and target saliency for subsequent target detection while compensating for the lack of semantic perception capability in traditional detection processes. LLaVA is a multimodal large language model integrating a vision encoder and a language model, with capabilities in image–text alignment, regional understanding, and natural language generation. Based on input images, it can automatically output structured semantic descriptions and perform language-level identification and analysis of quality-related regions.

In SemaFire-YOLO, the LLaVA module serves as an offline preprocessing component and implements three core functional steps as detailed below. This semantic enhancement is performed prior to both training and testing—all images in the dataset are preprocessed through the LLaVA pipeline before being fed into the detection network. During inference, the same enhanced images are used, ensuring consistency between training and testing phases. This offline preprocessing scheme is adopted mainly to avoid the high inference latency of the large multimodal model during online deployment, which is more compatible with the lightweight and real-time requirements of edge fire monitoring. For high-computing platforms, the pipeline can also be extended to online adaptive enhancement. To ensure fair comparison, all baseline models in Section 3.5 are also trained and evaluated on the same LLaVA-enhanced images. First, the original fire image is input to the LLaVA model, which generates natural language descriptions via the image–text cross-attention mechanism. These descriptions typically include structural information such as target clarity, contrast, brightness, and edge features, enabling the system to acquire high-level semantic priors.

Second, regular expressions are used to extract the above descriptions into structured regional information. Combined with open-source computer vision library (OpenCV) image processing tools, targeted image enhancement operations are performed based on this structured information. Brightness enhancement and histogram equalization are applied to dark regions; edge sharpening is conducted for low-contrast image edges; and local filtering or denoising is implemented for the entire image. All enhancement processes are driven by LLaVA-provided semantic guidance, exhibiting high selectivity and specificity.

Finally, the enhanced image—retaining its original size and format—is input to the backbone network (Backbone), which aligns with the input process of the standard YOLO model. Since the input image has undergone structural saliency enhancement and noise suppression, SemaFire-YOLO can obtain clearer fire boundaries and more stable response results in the subsequent inference phase. This combination of preprocessing and optimized inference performance significantly improves the model’s recognition capability for small targets, low-contrast regions, and dynamically diffusing targets.

The formal execution logic of the above pipeline is summarized in Algorithm 1.

Algorithm 1. LLaVA-driven semantic enhancement pipeline

Input: Original fire image I Output: Enhanced image I_enhanced 1: Feed I into the LLaVA model → Generate multi-dimensional semantic description D (covering subject identifiability, contrast, luminance, and edge characteristics) 2: Parse D via regular expressions → Extract structured attribute JSON T 3: Traverse each attribute field in T 4: if Luminance field matches “dark”/”low brightness” then Apply CLAHE (clipLimit = 2.0, tileGridSize = (8,8)) 5: else if Contrast/Edge field matches “blurred edge”/”low contrast” then Apply un sharp masking (sigma = 1.0) 6: else if Image quality field matches “noisy” then Apply bilateral filtering (d = 9, σ_color = 75, σ_space = 75) 7: end if 8: end for 9: Return I_enhanced

As a critical preprocessing component of the entire model, this module leverages the advantages of large language models in image semantic understanding without increasing the computational burden of the main network. Figure 2 shows the semantic guidance to feature extraction. It constructs a perception loop of “semantic guidance to image enhancement to feature extraction”, providing key support for developing a more intelligent and robust fire detection system.

2.3. HWD Feature Extraction Module

In traditional YOLO models, feature map downsampling is typically implemented via strided convolution or pooling. Although these methods can effectively compress spatial dimensions and improve computational efficiency, they exhibit limitations in fire and smoke detection tasks, including weak modeling capability for high-frequency detailed features and easy loss of boundary information. Specifically, when processing small targets or regions with blurred contours in early-stage fire scenarios, they often lead to reduced detection accuracy and insufficient target representation. To enhance the structural sensitivity of feature representation and the preservation capability of fine-grained details, this study introduces a HWD module [30], which is developed based on Haar wavelet transform, thus replacing the traditional downsampling structure in the Neck. As illustrated in Figure 3, the HWD comprises a lossless feature encoding block and a feature representation learning block.

It adopts 2D Haar wavelet transform to decompose the input feature map $X\in R^{H\times W\times C}$ into four directional frequency-domain components, including a low-frequency approximation component A and three high-frequency detail components {H, V, D}. The operation is mathematically expressed as follows:

$$\left\{\begin{array}{c}A=H_0^T\ast H_0\ast X\\ H=H_1^T\ast H_0\ast X\\ V=H_0^T\ast H_1\ast X\\ D=H_1^T\ast H_1\ast X\end{array}\right. \to F_{enc}=R^{\frac{H}{2}\times \frac{W}{2}\times 4C},$$

(1)

Among them, H₀ and H₁ represent the low-pass filter and high-pass filter of the Haar wavelet, respectively, and ∗ denotes the 2D convolution operation. This process effectively encodes spatial details (e.g., edges and textures) into the channel dimension while preserving feature integrity, thereby enhancing the structural expression capability of downsampling features.

To enhance feature discriminability and information aggregation efficiency after compression, the feature representation learning block further incorporates 1 × 1 convolution, BN, and the rectified linear unit (ReLU) activation function to perform nonlinear mapping and channel reorganization. The formulation is as follows:

$$F_{out}=RELU\left(BN\left(W_1\ast F_{enc}+b_1\right)\right), W_1=R^{C^{\prime }\times 4C\times 1\times 1},$$

(2)

among them, C′ denotes the number of output channels, which helps maintain dimensional consistency with subsequent layers of the YOLO backbone network.

In fire and smoke detection scenarios, the high-frequency components {H, V, D} generated by HWD exhibit strong responsiveness to local mutations at flame edges and diffusion features of smoke textures. The combination of these components enables the further construction of a detail-oriented directional feature enhancement representation:

$$G\left(F_i\right)={\nabla }_x{H}_i+{\nabla }_y{V}_i+{\nabla }_{xy}{D}_i,$$

(3)

Among them, ∇x, ∇y, and ∇xy denote the gradient operators in the horizontal, vertical, and diagonal directions, respectively. They significantly enhance the model’s capability to perceive the structure of irregular boundaries, fine-grained textures, and blurred targets. Additionally, Haar wavelets inherently possess multi-scale decomposition properties. The HWD module can extract detailed changes in flames and smoke at different scales in each downsampling stage. This complements the feature pyramid network (FPN) and path aggregation network (PAN) feature fusion mechanisms in the YOLO architecture, facilitating the capture of dynamic multi-scale target features during fire scenario evolution.

In summary, by constructing a frequency-domain-aware, structure-sensitive lossless downsampling mechanism, the HWD module effectively mitigates key issues of YOLO in fire and smoke detection, including weakened boundary information and small-target degradation. It provides a higher-quality feature foundation for subsequent network layers and acts as crucial structural support for achieving refined and real-time fire recognition.

2.4. A2C2f_ConvMod

The A2C2f module introduced in YOLOv12 offers advantages in enhancing feature expression capabilities, particularly excelling in global information fusion and cross-channel modeling. However, this module still centers on the self-attention mechanism: it first computes the matrix multiplication between queries (Q) and keys (K) to acquire global dependencies, then performs weighted summation of values (V). While this process can capture long-range feature correlations, its computational complexity is O(N²) (where N = H × W), resulting in low inference efficiency for high-resolution surveillance videos and thus making it difficult to meet the practical application requirements of real-time fire and smoke detection.

In addition, in fire and smoke detection scenarios, the self-attention mechanism exhibits limited capability in modeling aperiodic, fine-grained dynamic features (e.g., smoke diffusion and flame flickering) [31]. It is particularly prone to missed detections or insensitive responses for small-target fire spots and low-contrast smoke regions. Therefore, this study proposes replacing the self-attention structure in the original A2C2f module with a ConvMod [32], thereby constructing a more efficient and robust feature modeling method for fire and smoke. Figure 4 shows the comparison of ConvMod and the self-attention mechanism structure.

The core idea of this mechanism is to generate a weight matrix via deep convolution and modulate feature values using the Hadamard product. This achieves a feature weighting effect analogous to that of the attention mechanism while significantly improving the model’s inference efficiency. Specifically, the input feature map X∈R^H×W×C is transformed through two linear branches, yielding the following results respectively:

$$V=W_{2}X,$$

(4)

$$A={DWConv}_{k\times k}\left(W_{1}X\right),$$

(5)

among them, W₁ and W₂ are learnable linear mappings, and ${DWConv}_{k\times k}$ denotes a k × k depthwise convolution operation. This operation extracts local contextual correlations to form an adaptive spatial weight matrix A. Subsequently, feature modulation is achieved via element-wise multiplication:

$$Z=A⨀V.$$

(6)

In terms of structural integration, this study adopts a “reuse–replacement–fusion” reconstruction strategy for the original A2C2f module. First, the original module (CBS) combined with the convolutional Layer (Conv), BN and sigmoid linear unit (SiLU) is retained for basic feature extraction, and features are split into multiple parallel branches via the Split operation. Second, the self-attention module within the A2C2f is replaced with the ConvMod sub-structure, where each branch independently performs linear transformation, depthwise convolution, and modulation operations to capture fire and smoke features across different scales and regions. Finally, the outputs of all branches undergo non-linear transformation through the feedforward neural network (FNN) layer, with residual connections preserved:

$$X_{out}=X_{in}+MLP\left(Z\right).$$

(7)

Compared with the traditional self-attention mechanism, the convolutional modulation mechanism not only retains global modeling capability but also significantly reduces computational overhead. Furthermore, structural reconstruction is implemented for the A2C2f module. Figure 5 shows the architecture diagram of the A2C2f_ConvMod module. This change effectively addresses the limitations of A2C2f in high-resolution fire and smoke detection, including inadequate real-time performance and insufficient feature capture capability. It further provides a more efficient feature expression foundation for subsequent modules.

2.5. DyT-C3K2

In object detection tasks, the C3K2 structure is extensively employed in backbone network construction and intermediate feature representation enhancement, leveraging its stacked Bottleneck architecture and superior feature fusion capability. However, in the context of smoke and flame target detection in fire scenarios, the C3K2 module exhibits inherent performance limitations. Firstly, smoke and flame targets are characterized by complex attributes, including intense dynamic variation, blurred boundaries, and morphological diversity, which are prone to inducing significant deviations in feature distribution within deep neural networks. Secondly, the Bottleneck sub-module of the C3K2 module adopts BN for normalization. While this technique effectively stabilizes gradient propagation during model training, it is highly sensitive to small-batch inputs and relies on fixed statistical parameters during the inference phase. This characteristic readily causes feature drift and inconsistency in unstructured fire scenarios, thereby impairing the detection accuracy and robustness of the model.

To tackle the aforementioned issues, this paper introduces a novel normalization alternative mechanism, namely the DyT normalization strategy [33], and further constructs an improved module termed DyT-C3K2 on this basis, as illustrated in Figure 6. By retaining the fundamental architecture of the original C3K2, this module replaces the BN layer in the second convolutional unit (Conv2d + BN + SiLU) of its Bottleneck sub-module with the DyT strategy, thus establishing a lightweight normalization alternative path that dispenses with the reliance on statistical quantities while retaining the capability of dynamic range compression.

The core design philosophy of DyT originates from a critical rethinking of the operational mechanism of LN. Specifically, within deep neural networks, the input-output mapping of the LN layer follows an S-shaped curve, which is characterized by preserving linear response characteristics for intermediate input values while implementing compression and regulation for extreme values. Drawing on this insight, DyT realizes direct mapping of the input feature distribution by introducing learnable scaling parameter α and bias parameters γ, β, coupled with the bounded activation function tanh:

$$DyT\left(x\right)=\gamma ·\tanh \left(\alpha x\right)+\beta ,$$

(8)

among them, α denotes a globally learnable scalar that regulates the scaling magnitude of the input data, while γ and β represent per-channel learnable vectors, corresponding to the affine transformation terms in conventional normalization methods. In comparison with BN and LN, DyT achieves comparable regularization effects without the need to compute batch or sample-wise means and variances. Furthermore, this strategy autonomously learns the activation range compression policy through end-to-end training, thus demonstrating superior representational adaptability in complex scenarios.

The DyT-C3K2 module developed based on DyT normalization strategy is illustrated in Figure 7. In the baseline C3K2 architecture, the input features are first fed into a CBS module and then processed via parallel branches. Multiple C3k units, each equipped with a dual-path structure, are cascaded in series to implement multi-scale feature extraction. In this work, the DyT strategy is incorporated into the Bottleneck component of each C3k unit to replace the BN operation embedded in the second convolutional layer, thus enabling dynamic adaptive adjustment of the normalization mechanism throughout the feature enhancement phase.

The integration of the DyT strategy not only mitigates the model’s dependence on batch size and enhances its stability in the face of sparse samples or dynamic environments during practical deployment, but also circumvents the performance drift issue induced by BN’s reliance on fixed mean and variance during the inference phase.

Overall, the DyT-C3K2 module retains the inherent advantages of the C3K2 structure, including multi-branch parallelism and deep stacking, while introducing a lightweight, learnable normalization mechanism that dispenses with statistical estimation. This enables it to effectively enhance the model’s perception capability for complex target structures (e.g., smoke boundaries and flame textures) and simultaneously improve the adaptability and detection accuracy of the fire detection model in unstructured, high-noise scenarios.

3. Experimental Results

3.1. Experimental Setup and Metrics

3.1.1. Dataset

To address the research and application requirements of fire and smoke detection, this study constructs a Fire-Smoke dataset. Characterized by multi-scenario coverage, large-scale volume, and multi-scale target attributes, the dataset primarily collects images from public internet sources, which are subjected to rigorous manual screening, and is supplemented with self-captured data. Partial images of the classified dataset are provided in the Supplementary Materials. The dataset contains a total of 6376 smoke samples (Figure 8 shows the statistics of four main scenario categories, totaling 6376 images. In terms of data composition, about 85% of the samples are collected from public internet sources, and the remaining 15% are self-captured images from indoor and outdoor fire simulation scenarios. Geographically, it includes mountainous areas, hills, plains, urban areas, and other environments; temporally and spatially, it spans daytime/nighttime periods and indoor/outdoor spaces, involving specific locations such as forests and residential buildings (nighttime samples account for about 12% of the total dataset); additionally, it incorporates small-target smoke recognition tasks. In terms of weather conditions, it covers two common scenarios: sunny and cloudy.

Before dataset construction, we applied perceptual hashing (pHash) to remove duplicate and highly similar images with a similarity threshold of 0.9, to avoid data redundancy and overfitting. For image frames extracted from video sequences, we strictly split the dataset by video source—all frames from the same video belong to only one of the training, validation or test sets—to prevent data leakage caused by temporal correlation.

All samples have been manually annotated to accurately locate fire and smoke targets, providing reliable data support for the training and evaluation of fire and smoke detection models. The dataset is divided into training set, validation set and test set with a ratio of 8:1:1, corresponding to 5130, 666 and 580 samples respectively. All images in the dataset are preprocessed via the LLaVA semantic enhancement pipeline before training and evaluation, and all baseline models use the same enhanced images to ensure fair comparison. The dataset composition is shown in Figure 8.

3.1.2. Evaluation Metrics

• Precision

Precision quantifies the proportion of true positive samples among all instances predicted as positive by the model, and its mathematical expression is given by:

$$Precision= {\displaystyle \frac{TP}{TP+FP}},$$

(9)

specifically, TP is true positive, which denotes the count of positive samples that are correctly identified by the model. And FP is false positive, which represents the count of negative samples that are erroneously classified as positive instances.

2.

Recall

Recall quantifies the proportion of true positive samples that are correctly identified as positive by the model, and its corresponding mathematical expression is given by:

$$Recall= {\displaystyle \frac{TP}{TP+FN}},$$

(10)

among them, FN is false negative, which denotes the count of positive samples that are erroneously classified as negative instances by the model.

3.

F1

The F1 Score corresponds to the harmonic mean of precision and recall. It serves to balance the trade-off between these two metrics and is thus of particular significance in scenarios with class imbalance. Its mathematical formulation is given by:

$$F1= 2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}.$$

(11)

4.

Mean average precision

Mean average precision (mAP) is a widely adopted evaluation metric in the field of target detection and image detection. It is computed by first calculating the average precision (AP) for each category and then taking the arithmetic mean of all category-wise APs. Specifically, AP quantifies the area under the precision-recall (P-R) curve, where a higher value directly indicates superior detection performance of the model. The computational method for AP is defined as follows:

$$AP={\int }_0^{1}Precision\left(Recall\right).$$

(12)

The specific calculation of mAP allows for model performance evaluation under distinct intersection over union (IoU) thresholds. Specifically, mAP@50 denotes the mAP value when the IoU threshold is set to 0.5, where a detection result is deemed correct if the overlap ratio between the predicted bounding box and the ground truth box reaches 50%. By contrast, mAP@50:95 represents the mean value of mAP scores computed across a series of consecutive IoU thresholds (i.e., from 0.5 to 0.95 with an interval of 0.05). This metric is designed to comprehensively evaluate the model’s detection performance under varying precision constraints.

3.1.3. Experimental Environment

This experiment was implemented based on the PyTorch deep learning framework, targeting the object detection task. For hardware configuration, a high-performance NVIDIA GeForce RTX 3090 graphics card was employed, which is equipped with 24 GB of video memory and enables efficient processing of large-scale data computations. Additionally, a 12th-generation Intel Core i7-12700KF processor was selected, featuring a base clock frequency of 3.6 GHz and supporting multi-threaded parallel computing.

The hyperparameter configuration of the experiment was determined via iterative optimization, with key parameters set as follows: the optimizer is SGD with momentum of 0.937 and weight decay of 0.0005; a cosine annealing learning rate scheduler is adopted, with the initial learning rate configured to 0.0001 to balance the convergence rate and training stability of the model; the batch size was fixed at 64 to maximize the parallel computing capability of the GPU while circumventing video memory overflow; the IoU threshold of 0.6 is the positive sample assignment threshold during training, which is set slightly higher than the default value to filter low-quality positive samples with fuzzy boundaries and improve the localization accuracy of the model for fire-smoke targets; the number of training epochs was set to 200, and an early stopping strategy with patience = 20 was adopted to prevent overfitting and ensure stable convergence.

The detailed configuration of the experimental environment is shown in

Table 1. Summary of experimental environment configuration.

Item	Model/Configuration
Operating System	Windows 11
Programming Language	Python 3.9
CPU	12th Gen Intel(R) Core(TM) i7-12700KF
GPU	NVIDIA GeForce RTX 3090
GPU VRAM	24 GB
Framework	PyTorch 2.1
Learning Rate	0.0001
Batch Size	64
IoU Threshold	0.6
Epoch	200

.

3.2. Neck Comparison Experiment

To validate the effectiveness of the proposed HWD module in fire detection tasks, this study conducted comparative experiments between the baseline model YOLOv12n and its improved variant (YOLOv12n+HWD) integrated with the HWD module. All experiments were performed under the same experimental environment and with a unified input size of 640 × 640. The experimental results are summarized in

Table 2. Comparison of performance metrics between YOLOv12n and YOLOv12n+HWD in fire detection task.

Method	Size	$\mathit{m}\mathit{A}\mathit{P}\mathbf{50}$	$\mathit{m}\mathit{A}\mathit{P}$50-95	Recall	Precision
YOLOv12n	640 × 640	60.50	34.00	54.90	62.70
YOLOv12n+HWD	640 × 640	63.70	35.20	58.20	64.90

.

The experimental results demonstrate that the baseline model YOLOv12n already exhibits a certain capacity for flame and smoke detection. Nevertheless, its recall performance and detection boundary delineation still have considerable room for optimization. Following the integration of the HWD module, the improved model YOLOv12n+HWD achieved consistent performance gains across all evaluation metrics: specifically, its mAP50 and mAP50-95 values were increased to 63.70% and 35.20%, respectively, which attests to the model’s enhanced capabilities in target localization and multi-scale detection accuracy. Meanwhile, its recall and precision were elevated to 58.20% and 64.90%, respectively, indicating that the model not only improves the target detection rate but also effectively suppresses the false positive rate.

This performance improvement is primarily attributable to the structural perception capability of wavelet transform embedded in the HWD module during the downsampling process, which enables feature maps to better preserve the edge contours and texture details of flame and smoke targets. In fire-related scenarios, detection targets typically exhibit characteristics such as blurriness, background interference, and irregular morphology. Conventional downsampling methods are prone to discarding critical high-frequency information, whereas the HWD module compensates for this limitation via frequency-domain enhancement, thereby enhancing the detection robustness and boundary delineation capability of the model.

In conclusion, the HWD module can effectively enhance the model’s capacity for structural modeling of complex targets (e.g., flame and smoke) while preserving the lightweight architecture of the model, thereby providing a more reliable technical basis for real-time intelligent monitoring in fire scenarios.

To further validate the structural perception capability of the HWD module in fire detection tasks, this study performed activation heatmap visualization analysis on both the baseline YOLOv12n model and its improved variant (YOLOv12n+HWD), with the corresponding results illustrated in Figure 9. This figure features three groups of typical fire scene samples, covering complex scenarios including black smoke occlusion, thin smoke diffusion, and large-area forest fires. For each sample, the left column displays the original image, the middle column presents the activation map of the YOLOv12n model, and the right column shows the activation map of the YOLOv12n+HWD model.

From the visualization results, it can be explicitly observed that the activation regions of the YOLOv12n model exhibit a certain degree of offset and diffusion. Particularly in scenarios involving sparse smoke or blurred boundaries, the model fails to accurately localize the primary detection targets. By contrast, following the integration of the HWD module, the activation heatmaps of YOLOv12n+HWD yield more concentrated response regions and clearer boundary contours across all fire scene categories. Specifically, in the black smoke fire scenario (first row), YOLOv12n+HWD is capable of precisely focusing on the edge contours of flames and dense smoke; in the thin smoke scenario (second row), the HWD-integrated model significantly enhances the response sensitivity to low-contrast target regions; in the large-area fire scenario (third row), the improved model can effectively cover multiple key combustion areas while suppressing background interference.

These results demonstrate that the HWD module possesses distinct advantages in preserving high-frequency texture and edge information. It significantly enhances the model’s capability for structural modeling of complex targets (e.g., flame and smoke), while strengthening the perception of weak targets and local features—thereby endowing the model with higher accuracy in practical fire monitoring scenarios involving low contrast and dynamic fire conditions.

3.3. Attention Mechanism Comparison Experiment

To validate the effectiveness of the proposed A2C2f_ConvMod module in the backbone network, this study designed comparative experiments based on the YOLOv12 backbone framework, including the baseline A2C2f structure and three improved variants. These variants are A2C2f-channel additive self-attention (CASA) [34], A2C2f-spatial strip attention (SSA) [35], and A2C2f-ConvMod [32]. All models were trained and evaluated under a unified input resolution (640 × 640) and consistent training configurations, with the experimental results summarized in

Table 3. Performance comparison of A2C2f modules improved with different attention mechanisms in fire detection task.

Method	Size	$\mathbf{m}\mathbf{A}\mathbf{P}\mathbf{50}$	$\mathbf{m}\mathbf{A}\mathbf{P}$50-95	Recall	Precision
A2C2f	640 × 640	60.50	34.00	54.90	62.70
A2C2f-CASA [34]	640 × 640	61.20	34.10	54.00	72.90
A2C2f-SSA [35]	640 × 640	62.30	34.80	54.60	72.50
A2C2f-ConvMod [32]	640 × 640	62.70	35.60	53.40	71.00

.

As illustrated in Table 3, following the integration of the channel additive attention mechanism into the A2C2f-CASAttention module, the model’s mAP50 increases from 60.50% to 61.20%, while its precision is significantly elevated to 72.90%—attesting to the module’s robust feature screening capability. Nevertheless, its recall decreases to 54.00%, indicating that this mechanism entails a certain risk of missed detection in dynamic fire and smoke regions. By integrating the spatial strip attention mechanism, the A2C2f-SSA module further enhances the model’s capability for modeling local spatial directional information, with its mAP50, mAP50-95 and precision increased to 62.30%, 34.80% and 72.50%, respectively, thus yielding more balanced overall performance. Nevertheless, given that its modeling strategy remains focused on static features, the module’s response to the morphological variation in irregular flames and the dynamic diffusion of smoke is still constrained. By contrast, the A2C2f-ConvMod module employs a proposed convolution modulation mechanism that integrates depthwise convolution with Hadamard feature modulation strategies. It achieves mAP50 and mAP50-95 values of 62.70% and 35.60%, respectively, outperforming all comparative models. Although its recall exhibits a slight decrease compared to the baseline model (dropping to 53.40%), its precision remains at a high level (71.00%). This indicates that the proposed mechanism can effectively suppress false positives while enhancing the model’s focus on response features of key regions. The significant performance gain is attributable to the ConvMod module’s efficient local context modeling capability, which endows it with stronger structural perception, particularly in edge-blurred regions and fire-smoke transition zones.

3.4. Ablation Experiment Verification

To validate the effectiveness of the three proposed structural improvements in fire detection tasks, this study conducted an ablative experiment on the baseline YOLOv12n model by evaluating the individual contributions of three components: the frequency-domain downsampling module (HWD), the A2C2f replacement mechanism (ConvMod), and the normalization replacement module (DyT-C3K2). The experimental results are summarized in

Table 4. Ablation experiment of each improved module for YOLOv12n.

Method	LLaVA	HWD	A2C2f_ConvMod	DyT-C3K2	$\mathbf{m}\mathbf{A}\mathbf{P}\mathbf{50}$	Param (M)	GFLOPs (G)
YOLOv12n					59.10	2.56	6.3
YOLOv12n-LLaVA	* √				60.50	2.56	6.3
YOLOv12n-LLaVA-HWD	√	√			63.70	2.46	6.2
YOLOv12n-LLaVA-A2C2f_ConvMod	√		√		62.70	2.52	6.3
YOLOv12n-LLaVA-HWD-A2C2f_ConvMod	√	√	√		64.20	2.38	6.1
YOLOv12n-LLaVA-DyT-C3K2	√			√	61.40	2.58	6.5
All	√	√	√	√	64.30	2.46	6.4

* √ means the corresponding module is adopted.

.

When only the LLaVA semantic enhancement is applied to the original images, the mAP50 increases from 59.10% to 60.50% (+1.40 pp), confirming that the semantic guidance preprocessing effectively improves the feature saliency of fire and smoke targets. This gain is achieved without any change to the network architecture (Params and GFLOPs remain identical at 2.56M and 6.3G, respectively), as LLaVA operates as an offline preprocessing component.

When the HWD module is further integrated, the mAP50 increases to 63.70% (+3.20 pp over the LLaVA-only baseline), demonstrating that Haar wavelet transform-based downsampling can effectively preserve edge and texture details, thereby improving the model’s perceptual performance for flame boundaries and smoke textures.

After substituting A2C2f with the ConvMod mechanism, the mAP50 climbs to 62.70% (+2.20 pp over the LLaVA-only baseline). This indicates that the large-kernel convolution-driven convolution modulation mechanism not only possesses more efficient spatial context modeling capability but also significantly reduces inference complexity.

The DyT-C3K2 structure yields a relatively limited improvement in detection accuracy, with the mAP50 reaching 61.40% (+0.90 pp over the LLaVA-only baseline). Nevertheless, this module exhibits stronger normalization robustness in unstructured and high-noise scenarios, which contributes to enhancing the model’s stability during the inference phase.

When the HWD and ConvMod modules are further combined, the model’s mAP50 rises to 64.20%, which attests to the complementary effects of the two modules in structural detail preservation and spatial context modeling. With the integration of all four components, the model achieves the highest mAP50 of 64.30%, representing a 5.20 pp improvement over the original YOLOv12n (59.10%). Notably, the parameter count of SemaFire-YOLO (2.46M) is even lower than that of the baseline (2.56M), primarily because the ConvMod mechanism replaces the self-attention structure with more parameter-efficient depthwise convolution and Hadamard modulation. The HWD module also maintains parameter efficiency through its fixed Haar wavelet basis functions.

These results validate the effectiveness of the proposed frequency-domain perception, normalization replacement, lightweight modeling, and semantic enhancement mechanisms for complex fire detection tasks. The collaborative deployment of the four components can significantly boost the model’s comprehensive performance in small target detection, boundary recognition, and uncertain environment adaptation.

3.5. Comparison Experiment

To validate the performance advantages of the proposed SemaFire-YOLO model in fire detection tasks, this study conducted a systematic comparative experiment between the proposed model and state-of-the-art lightweight object detection models, including YOLOv5n [36], YOLOv6n [37], YOLOv8n [38], YOLOv10 [39], YOLOv11n [40], and YOLOv12n [28]. All models were trained and evaluated on the fire image dataset under a unified training strategy and a consistent input size of 640 × 640. The experimental results are summarized in

Table 5. Comparison of fire detection performance metrics between SemaFire-YOLO and YOLO series models.

Method	Size	$\mathbf{m}\mathbf{A}\mathbf{P}\mathbf{50}$	$\mathbf{m}\mathbf{A}\mathbf{P}$50-95	Recall	Precision
YOLOv5n [36]	640 × 640	63.50	35.70	55.20	69.80
YOLOv6n [37]	640 × 640	63.00	35.30	54.50	68.00
YOLOv8n [38]	640 × 640	62.30	35.90	56.60	68.50
YOLOv10 [39]	640 × 640	60.10	33.70	56.30	65.80
YOLOv11n [40]	640 × 640	63.70	36.30	54.50	72.10
YOLOv12n [28]	640 × 640	60.50	34.00	54.90	62.70
SemaFire-YOLO	640 × 640	64.30	35.40	56.60	69.60

.

In terms of overall performance metrics, SemaFire-YOLO achieves an mAP50 of 64.30%, the highest among all compared models. It improves over the architectural baseline YOLOv12n (60.50%) by 3.80 pp and over YOLOv11n (63.70%) by 0.60 pp. We acknowledge that SemaFire-YOLO’s mAP50-95 (35.40%) is marginally lower than that of YOLOv11n (36.30%), YOLOv8n (35.90%), and YOLOv5n (35.70%), which is primarily due to two reasons. First, fire and smoke targets inherently have irregular, fuzzy boundaries without clear physical edges, making high-IoU precise localization inherently difficult. Second, the HWD module emphasizes preserving high-frequency structural features that benefit coarse localization (IoU = 0.5), while its contribution to precise bounding box regression at stricter IoU thresholds is relatively limited. Nevertheless, in fire detection scenarios, missing a fire target carries far more severe consequences than having a slightly less precise bounding box. SemaFire-YOLO achieves the highest recall (56.60%) among all compared models, indicating a lower missed detection rate—a critical requirement for early fire warning systems. Furthermore, SemaFire-YOLO achieves a favorable balance between precision (69.60%) and recall, yielding the highest F1 score among the lightweight models. The observed performance enhancement is attributable to three key structural optimizations. First, the integrated HWD module preserves the high-frequency boundary and texture features of flames and smoke during the backbone downsampling phase, thereby enhancing the model’s perceptual capability for small and blurred targets. Second, the ConvMod spatial modeling mechanism realizes a lightweight alternative for contextual modeling via depthwise separable convolution, which effectively suppresses redundant interfering features. Finally, the normalization replacement strategy in the DyT-C3K2 module enhances the network’s adaptability to complex data distributions and dynamic scenarios, improving the model’s robustness during the inference phase.

The visualized detection results in Figure 10 further validate the conclusions of the aforementioned quantitative experiments. Compared with the benchmark model YOLOv12n, SemaFire-YOLO exhibits superior target perception capability and localization accuracy across diverse fire scenarios.

Specifically, in the forest fire image of the first row, YOLOv12n suffers from insufficient boundary coverage and low confidence in flame detection. By contrast, SemaFire-YOLO can accurately segment multiple flame regions, with detection boxes closely matching the actual edges and significantly improved confidence. This reflects its advantage in modeling high-frequency edges and textures, which is precisely attributed to the adequate preservation of structural information by the HWD module.

In the low-contrast fire image of the second row, YOLOv12n fails to detect the target, whereas SemaFire-YOLO accurately identifies the flame region by virtue of its enhanced perception capability for small and blurred targets. This indicates that the model possesses more robust response capability in weak-feature scenarios. Such performance is closely associated with the lightweight context-aware mechanism constructed by ConvMod, which can effectively strengthen local feature representation and suppress background interference.

In the complex highway fire scenario of the third row, although YOLOv12n recognizes smoke, it fails to detect the flame region. In contrast, SemaFire-YOLO simultaneously detects both smoke and flame targets, providing accurate bounding boxes and high confidence. This demonstrates the model’s strong generalization ability in multi-target and multi-modal scenarios, and also reflects the practical effect of the normalization replacement strategy in the DyT-C3K2 module in enhancing network stability and dynamic adaptability.

4. Conclusions

This study proposes a lightweight fire detection model, namely SemaFire-YOLO, which integrates frequency-domain enhancement, semantic perception, and normalization optimization mechanisms. Aiming at the inherent limitations of existing lightweight fire detection methods in small target modeling, dynamic scene adaptation, and inference stability, this study conducts systematic improvements from multiple dimensions, including input preprocessing optimization, network structural design, and feature normalization regulation. The core innovation of the model lies in four proposed modules, as elaborated below:

First, the LLaVA semantic guidance mechanism innovatively introduces a large language model into the fire detection pipeline. By leveraging image semantic understanding and structural enhancement to guide preprocessing operations (e.g., brightness enhancement, edge sharpening, and noise reduction), it significantly improves the separability and feature saliency of weak target regions.

Second, the HWD module implements frequency-domain downsampling via Haar wavelet transform, which effectively preserves high-frequency details (e.g., flame edges and smoke textures) that are critical for fire detection.

Third, the ConvMod module constructs an efficient contextual modeling pathway using large-kernel depthwise convolution and Hadamard modulation. This design enhances local feature perception while reducing computational overhead, realizing a trade-off between performance and efficiency.

Fourth, the DyT-C3K2 module replaces the traditional BN with a DyT normalization strategy, which improves the model’s normalization stability under unstructured and dynamic input data distributions.

Extensive experiments are conducted on the multi-scenario and multi-scale Fire-Smoke dataset. Experimental results demonstrate that SemaFire-YOLO achieves an mAP@0.5 of 64.30%, outperforming YOLOv12n by 3.8 pp and achieving competitive or superior mAP@0.5 compared with state-of-the-art lightweight detection models (e.g., YOLOv5n, YOLOv8n, and YOLOv11n). Although its mAP@0.5:0.95 is marginally lower than that of YOLOv11n, SemaFire-YOLO achieves the highest recall among all compared models, which is of paramount importance for early fire detection where missed detection carries severe safety consequences. The model maintains a lightweight architecture with competitive inference speed, confirming its suitability for real-time deployment. Visualization analysis further validates its improved detection capability and structural detail preservation advantages in complex fire scenarios involving dynamic spread, low contrast, and multi-target coexistence. However, this study has several limitations. First, the robustness of SemaFire-YOLO has not been systematically evaluated under extreme scenarios such as fog, strong light, reflections, and smoke-like interferences. Future work will construct a multi-scenario benchmark for comprehensive robustness validation. Second, the LLaVA enhancement pipeline relies on rule-based regex parsing, which may miss edge cases; learnable prompt adaptation will be explored. Third, the current dataset covers only fire and smoke; extending to multi-hazard detection will broaden applicability.