FireCLIP: Enhancing Forest Fire Detection with Multimodal Prompt Tuning and Vision-Language Understanding

Abstract

Forest fires are a global environmental threat to human life and ecosystems. This study compiles smoke alarm images from five high-definition surveillance cameras in Foshan City, Guangdong, China, collected over one year, to create a smoke-based early warning dataset. The dataset presents two key challenges: (1) high false positive rates caused by pseudo-smoke interference, including non-fire conditions like cooking smoke and industrial emissions, and (2) significant regional data imbalances, influenced by varying human activity intensities and terrain features, which impair the generalizability of traditional pre-train–fine-tune strategies. To address these challenges, we explore the use of visual language models to differentiate between true alarms and false alarms. Additionally, our method incorporates a prompt tuning strategy which helps to improve performance by at least 12.45% in zero-shot learning tasks and also enhances performance in few-shot learning tasks, demonstrating enhanced regional generalization compared to baselines.

1. Introduction

Forest fires represent a catastrophic phenomenon with far-reaching implications for the global ecosystem, economy, and human well-being. The impact of these uncontrolled blazes extends from the loss of biodiversity and habitats to the exacerbation of climate change through the release of vast amounts of carbon dioxide into the atmosphere [1,2]. From an economic perspective, forest fires impose significant financial burdens in several respects, such as the allocation of resources to firefighting efforts, the reconstruction of affected areas, and the loss of forest resources [3,4]. Moreover, the societal impact is profound, with the potential for loss of human life and the displacement of communities [5,6]. The destructive nature of forest fires necessitates robust and proactive detection systems to mitigate their occurrence and impact [7,8].

In the context of forest fire detection, smoke is typically detected before the flames become visible, making it a crucial indicator for early detection systems. Therefore, current mainstream technologies often focus on identifying the occurrence of forest fires by observing smoke [9,10]. Traditional detection methods rely on smoke detectors to sense smoke produced by combustion [11]. However, in open outdoor environments, smoke tends to disperse rapidly and is challenging to monitor effectively. Additionally, fine particulate matter such as dust in the atmosphere may interfere with the proper functioning of detectors, leading to reduced detection efficiency and accuracy. In contrast, image-based smoke recognition technology, with its broad detection range and cost-effectiveness, has been widely applied in the field of forest fire early warning detection [12]. Traditional image processing techniques for smoke detection have relied on features such as color, texture, and motion to distinguish smoke from the background [13,14]. However, these methods are often limited by their inability to generalize across varying environmental conditions and the diversity of smoke appearances.

The evolution of forest fire detection methods displays both technological progress and persistent challenges. Early approaches relied on physical patrols and lookout stations [15], later supplemented by sensor networks and satellite monitoring. The advent of deep learning brought significant breakthroughs through architectures like DeepSmoke’s CNN framework [16] and Xu et al.’s dual-stream network combining pixel-level and object-level analysis [17]. Subsequent innovations include Yuan et al.’s waveform neural network for smoke concentration prediction [18], Azim’s fuel classification model for emergency planning [19], and Qiang et al.’s TRPCA-enhanced video analysis [20]. While these methods improved detection accuracy, practical deployment revealed limitations in handling geographic variations and pseudo-smoke interference. The YOLO-PCA algorithm [21] addressed efficiency through principal component analysis, and FireFormer [22] demonstrated exceptional performance in specific WUI environments. Nevertheless, cross-regional adaptation remains challenging due to data distribution shifts.

The deep learning revolution has transformed wildfire detection through enhanced feature extraction capabilities. Modern architectures overcome manual feature engineering limitations via end-to-end learning [23,24], with transfer learning enabling effective knowledge transfer from pre-trained models [25]. Notable advancements include FireFormer [22], which integrates Swin Transformer’s multi-scale hierarchical representation [26] for smoke pattern recognition in wildland–urban interface environments. Nevertheless, three fundamental challenges persist.

First, current detection systems demonstrate heightened sensitivity to pseudo-smoke phenomena—including cooking smoke and industrial emissions—resulting in excessive false positives. This limitation stems from disproportionate emphasis on feature localization over comprehensive risk assessment. Second, smoke classification research predominantly operates within constrained datasets exhibiting low inter-class confusion, creating significant gaps in adversarial robustness research. Third, practical deployment encounters data scarcity challenges due to wildfire unpredictability and geographical diversity, resulting in dispersed feature distributions that complicate model adaptation [27].

Few-shot learning emerges as a promising paradigm for addressing data paucity through prior knowledge utilization from large labeled datasets [28]. However, it must resolve two critical challenges: category shifts between training and testing distributions and extreme label scarcity [29]. While conventional mitigation strategies like data augmentation provide partial solutions, they fail to prevent model overfitting (where a model memorizes limited examples instead of learning general patterns) when fine-tuning with limited samples.

Recent advances in prompt engineering offer novel adaptation strategies for large pre-trained models. By integrating task-specific prompts—either through linguistic instructions [30,31] or learnable embeddings [32]—models can better leverage pre-acquired knowledge for downstream tasks.

Building on CLIP’s vision-language alignment capabilities [33], recent prompt tuning methods have enhanced model adaptability. CoOp [34] introduced learnable text prompts, while MaPLe [35] extended this to multi-modal prompting through deep visual–linguistic adaptation. Subsequent innovations like PromptSRC’s self-regularization [36] and ProMetaR’s meta-regularization [37] improved generalization through the explicit prevention of representation overfitting. Architectural advancements include DePT’s feature space decoupling [38] and MetaPrompt’s domain-invariant tuning [39], which address the base–new tradeoff through meta-learning strategies. These developments establish prompt engineering as a powerful paradigm for few-shot adaptation—a critical requirement for wildfire detection systems facing data scarcity.

The prompt tuning paradigm demonstrates particular promise through its parameter-efficient adaptation mechanism, achieving superior few-shot performance compared to conventional transfer learning [32].

In this work, we introduce the FireCLIP model for forest fire smoke detection that integrates prompt tuning (adapting models via task-specific input instructions rather than full retraining) to address the challenges of transfer learning in new regions and few-shot learning due to data scarcity. FireCLIP is based on the large-scale vision-language pre-trained model CLIP [33] and is adapted to the specific task of smoke detection through prompt tuning. This method aims to enhance the model’s ability to generalize from a small number of examples and to adapt to the unique characteristics of smoke in a new domain.

Key Innovations:

• Identification of dual challenges in forest fire detection: pseudo-smoke false positives and regional data imbalance.
• Empirical validation of vision-language prompt learning’s effectiveness in addressing these challenges through adaptive feature extraction and cross-modal understanding.
• Contribution of a novel smoke recognition benchmark dataset featuring high inter-class confusion, specifically designed to evaluate model robustness against pseudo-smoke phenomena.

2. Materials and Methods

2.1. Wildland–Urban Interface Smoke Alarm Dataset (WUISAD)

Foshan City, located in the central-southern part of Guangdong Province, China, covers a total area of 3797.72 square kilometers. The city is one of the most important bases for manufacturing in the country, with a highly developed industrial system. Geographically, the western part of Foshan City is primarily characterized by low hills and hillocks. In terms of climate, the city is located within the subtropical monsoon climate zone, with an average annual temperature of about 22.5 °C and an average annual precipitation of approximately 1681.2 mm. Such climatic conditions are extremely conducive to the growth and flourishing of forests and various types of vegetation. Consequently, in areas where forests and urban residential or industrial zones are adjacent, a typical wildland–urban interface (WUI) landscape has been formed.

We used high-resolution cameras equipped with conventional detection algorithms to capture images exhibiting smoke characteristics. To obtain a diverse and representative dataset, we deployed five long-range surveillance cameras across various regions of Foshan City. Each camera is capable of operating within a radius of 0–8 km. Equipped with both visible-light and infrared imaging technologies, the cameras feature DC brushless motors for rotation and a visible focal length lens with an 8–560 mm range, enabling up to 70× zoom capabilities. With a resolution exceeding eight megapixels and a maximum image resolution of 3840 × 2160 pixels, these cameras are ideally suited for remote monitoring applications, particularly in protected areas such as forest fire prevention zones.

After excluding images without distinct smoke features, we meticulously categorized the remaining samples into five types: Field Burning and Wildfires, Illuminated by Light, Industrial Emissions, Mist and Water Vapor, and Village Cooking Smoke, as shown in

Table 1. Various examples of smoke alarm images. All labels are concatenated with the prompt before being input into the text encoder of CLIP. This allows the model to better differentiate between various types of samples based on text labels rather than abstract numerical labels.

Field Burning and Wildfires	Illuminated by Light	Industrial Emissions	Mist and Water Vapor	Village Cooking Smoke

. The “Field Burning and Wildfires” class focuses on detecting agricultural burning activities, which are not major wildfire incidents themselves, but represent critical fire-ignition behaviors demanding early identification for effective forest fire prevention.

Furthermore, we subdivided these samples into five sub-datasets based on geographic regions. Images from Datasets A to E were collected by multiple high-definition cameras located in various areas of Foshan City. The specific locations of each camera and the distribution of images within the datasets are shown in

Table 2. Distribution of images in each dataset.

Dataset	Location	Field Burning and Wildfires	Illuminated by Light	Industrial Emissions	Mist and Water Vapor	Village Cooking Smoke
A	Yanghe Town	5689	432	257	136	1435
B	Mingcheng Town	2037	204	3880	210	73
C	Sanshui District	728	20	418	128	171
D	Genglou Town	760	8	0	108	100
E	Sanshan Forest Park	68	14	38	127	21

.

Given that Datasets A and B are both of considerable size, and that their combination can form a more evenly distributed dataset, we decided to merge these two datasets into a single unified dataset. This integration aimed to construct a pre-training dataset that was more balanced across various label classifications, thereby enhancing the model’s generalization capability and performance.

Due to the distinct characteristics of the remaining three datasets and the relatively limited amount of available data, we decided to utilize them as datasets for downstream tasks. This approach was designed to simulate the practical challenges that may be encountered during the generalization process of pre-trained models, thereby assessing and enhancing the model’s adaptability and application potential in data-scarce environments.

2.2. Forest Fire Detection System

Figure 1 presents a forest fire detection system that detects and classifies potential fire threats through a series of steps. The system initially utilizes high-definition surveillance cameras deployed in areas prone to fires to capture real-time environmental images.

These cameras integrate cutting-edge imaging technology, enabling the continuous detection of smoke features. After capturing the images, the system initiates a preliminary screening process that automatically identifies and locates potential problem areas, such as the presence of smoke, through image processing algorithms. This recognition process is executed by analyzing patterns and features related to smoke within the images. Once the system detects smoke, it further classifies the smoke based on risk levels, distinguishing between high-risk situations that may indicate an imminent forest fire and low-risk situations.

The classification phase is a complex and critical step. Traditionally, fine-tuning pre-trained models on large datasets specific to the task has been employed for such tasks. Fine-tuning (model tuning) [40] refers to the process of adjusting and optimizing the parameters of a pre-trained model to improve its performance on a specific task or dataset. This typically involves fine-tuning certain layers or hyperparameters of the model, using a smaller, task-specific dataset, to adapt the model to the new application. Tuning can include techniques such as adjusting learning rates, regularization terms, or the number of training epochs. The goal of model tuning is to enhance the model’s generalization ability and accuracy without overfitting to the new data. However, this approach often performs poorly in scenarios where data is scarce or unevenly distributed across regions.

The system can also adopt a prompt tuning strategy, an innovative method to address the aforementioned challenges. Prompt tuning enables a model to adapt to new tasks and datasets with minimal additional training by introducing task-specific prompts into the input of a pre-trained model. This method is particularly effective in data-limited contexts as it allows the model to generalize effectively by leveraging the knowledge encoded in the pre-trained parameters.

The advantages of prompt tuning over traditional fine-tuning are multifaceted. Firstly, it significantly reduces the computational resources required, as the model does not need to relearn all parameters. Secondly, it enhances the model’s adaptability, enabling it to quickly adjust to new scenarios or regions with varying environmental conditions. Lastly, by utilizing region-specific prompts, the system can deploy a single model to handle forest fire detection tasks across multiple areas.

2.3. Multi-Modal Prompt Tuning (MaPLe)

MapLe [35] is an improvement over the CLIP framework, incorporating learnable prompts into both the visual and linguistic branches of CLIP to enhance the model’s generalization capabilities. The framework is shown in Figure 2.

Specifically, MaPLe introduces learnable tokens into the linguistic branch of the CLIP model. The input embeddings take the form of $([P_1,P_2,\dots ,P_b,W_0])$, with $W_0$ corresponding to fixed input tokens. Additional learnable tokens are integrated at each transformer block of the language encoder ($L_i$) up to a specified depth J. The equations are as follows:

$$[\_\_,W_i]=Li([P_{i-1},W_{i-1}])\mathrm{for}i=1,2,\dots ,J,$$

(1)

$$[P_j,W_j]=Vj(P_{j-1},W_{j-1}])\mathrm{for}j=J+1,\dots ,K.$$

(2)

Here, $[·,·]$ denotes the concatenation operation. After the Jth transformer layer, subsequent layers process the previous layer’s prompts and compute the final text representation z, which is projected into a common $V-L$ latent embedding space via TextProj,

$$z=\mathrm{TextProj}(w_K^N),z\in {\mathbb{R}}^{d_{vl}}.$$

(3)

Similarly to language prompt learning methods, learnable vision prompts (denoted $(\{P_i\in {\mathbb{R}}^{d_v}\})$) are introduced within the visual branch of the CLIP model, with new learnable tokens being integrated deeper into the transformer layers of the image encoder (Vi) up to depth J.

The equations are presented as follows:

$$[c_i,E_i,\_\_]=Vi([c_{i-1},E_{i-1},P_{i-1}])\mathrm{for}i=1,2,\dots ,J,$$

(4)

$$[c_j,E_j,P_j]=Vj([c_{j-1},E_{j-1},P_{j-1}])\mathrm{for}j=J+1,\dots ,K,$$

(5)

$$x=\mathrm{ImageProj}(c_K)x\in {\mathbb{R}}^{d_{vi}}.$$

(6)

Considering that a multimodal approach is essential for prompt tuning, where both the vision and language branches of CLIP are simultaneously adapted to achieve comprehensive context optimization. MaPLe proposes a multimodal prompting method that tunes the vision and language branches of CLIP together by sharing prompts across both modalities. Language prompt tokens are introduced in the language branch up to the Jth transformer block, akin to deep language prompting. To ensure synergy between $V-L$ prompts, vision prompts “P” are obtained by projecting language prompts P via a vision-to-language projection, referred to as the $V-L$ coupling function $F(·)$.

The equations are given by

$$\widehat{P_k}=F_k(P_k).$$

(7)

Here, $F(·)$ is a linear layer that maps $d_l$-dimensional inputs to $dv$. This serves as a bridge between the two modalities, encouraging the mutual propagation of gradients.

The full equations are as follows:

$$[c_i,E_i,\_\_]=Vi([c_{i-1},E_{i-1},{\widehat{F}}_{i-1}(P_{i-1})])\mathrm{for}i=1,\dots ,J,$$

(8)

$$[c_j,E_j,\widehat{P_j}]=Vj([c_{j-1},E_{j-1},{\widehat{P}}_{j-1}])\mathrm{for}j=J+1,\dots ,K,$$

(9)

$$x=\mathrm{ImageProj}(c_K).$$

(10)

Through the aforementioned methodology, MaPLe effectively captures the interaction between the visual and linguistic branches when fine-tuning pre-trained CLIP models, thereby improving the model’s generalization capabilities to novel categories, new target datasets, and unseen domain shifts.

3. Results

3.1. Benchmark Setting

To evaluate the effectiveness of FireCLIP, this study selected FireFormer [22], ConvNeXt [41], and EfficientNetV2 [42] as the primary baseline models. FireFormer has demonstrated outstanding performance in relevant domains. ConvNeXt, an optimized version of traditional convolutional neural networks, integrates design principles from Vision Transformers to enhance model expressiveness. EfficientNetV2, a highly efficient convolutional neural network, achieves an optimal balance among network depth, width, and resolution, exhibiting superior computational efficiency and feature representation capabilities. Through comparative analysis with these baseline models, this study aims to comprehensively assess the advantages and application potential of FireCLIP.

The experimental setup is as follows:

• Pre-training: We deployed the baseline models on a pre-training dataset for pre-training, and compared the performance and efficiency of the baseline models versus FireCLIP.
• Zero-shot Transfer: We directly applied the pre-trained model to the dataset of a downstream task to observe the differences in generalization ability between baselines and FireCLIP methods.
• Few-shot Learning: We conducted few-shot learning with the pre-trained model on the dataset of a downstream task. The learned model was then tested on both the pre-training dataset and the downstream task dataset to assess the performance of baselines and FireCLIP in 4-, 8-, 16-, and 32-shot settings.

For fair comparison, all baseline methods underwent identical training settings during both the pre-training and few-shot learning phases. Model versions are specified in

Table 3. Comparison of models, parameters, and FPS. The FPS in the table indicates the number of images per second that the model is able to process during evaluation. “ViT-B/16” for FireCLIP refers to its visual encoder architecture and 154 M represents the total parameters including the language encoder.

Model	Variant	Params	FPS
FireCLIP	ViT-B/16	154 M	91
Fireformer	Swin-L	197 M	58
ConvNeXt	ConvNeXt-L	198 M	58
EfficientNetV2	EffNetV2-L	119.5 M	101

.

3.2. Evaluation Setting

To address the problem of data balance and the measurement of single-category precision in the classification task, four commonly used evaluation metrics in classification task were adopted to comprehensively evaluate the performance of model, as defined in Equations (11)–(14):

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+FN+FP+TN}},$$

(11)

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}},$$

(12)

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}},$$

(13)

$$F1\mathrm{score}={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}.$$

(14)

where TP is true positives with a positive label and positive model prediction, TN is true negatives with a negative label and negative model prediction, FP is false positives with positive prediction but a negative label, and FN is false negatives with negative prediction but a positive label.

3.3. Implementation Details

We conducted experiments on two datasets, namely the WUISAD and FASDD [43] datasets. FASDD is a large-scale, multi-source collection comprising 122,634 samples of fire-related phenomena (flames and smoke) and non-fire distractors. This community-driven dataset features crowdsourced annotations and multimodal visual data collected from three distinct visual sensor modalities: surveillance cameras, unmanned aerial vehicles (UAVs), and satellite imaging. We performed binary classification tasks on the two subsets, FASDDCV (surveillance camera-captured data) and FASDDUAV (UAV-acquired imagery), to distinguish between the presence and absence of fire and smoke.

We employed a few-shot training strategy in all experiments, with random sampling for each class. All experiments were conducted on a single A100 80G GPU, with the batch size set to 32 and the image size adjusted to 224 × 224. The experimental results for all few-shot learning tasks represent the average values after three samplings. We performed a five-class classification task on the WUISAD dataset, training all models for 100 epochs. Conversely, for the FASDD dataset, we conducted a two-class classification task, limiting the training to five epochs. This configuration was adopted due to the significantly higher complexity and classification difficulty inherent in the WUISAD dataset compared to FASDD.

Prompt tuning was applied to a pre-trained ViT-B/16 CLIP model, which was trained on the WIT dataset, containing 400 million (image, text) pairs. We set prompt depth J to 9 and the language and vision prompt lengths to 2. We initialized the language prompts with the pre-trained CLIP word embeddings of the template ‘a photo of a’. Other settings are the same as in MaPLe.

3.4. Performance Comparison

3.4.1. Pre-Training

This section investigates the learning capacities of the baseline models and FireCLIP across datasets from various regions. During training, the baseline models fine-tune all parameters, whereas FireCLIP employs a more parameter-efficient approach by freezing the pre-trained parameters of CLIP and solely learning prompts. Consequently, there is a substantial difference in the number of parameters being learned, with the baseline models utilizing at least 119.5 million parameters compared to FireCLIP’s 3.55 million parameters.

Table 4. Binary classification accuracy (%) of each model after pre-training on the FASDD dataset.

Dataset	FireCLIP	Fireformer	ConvNeXt	EfficientNetV2
FASDDCV	99.52	98.51	98.64	97.82
FASDDUAV	93.56	91.75	91.93	92.96

presents the pre-training results of various methods on the FASDD dataset. Despite being trained for only five epochs, all methods demonstrated commendable performance. These results further underscore the relatively lower discriminative complexity of the FASDD dataset.

According to Figure 3, the FireCLIP and baseline models demonstrate comparable performance across various datasets of WUISAD. This suggests that integrating CLIP’s prompt tuning method can achieve performance levels similar to the fine-tuned FireFormer, even with a significant reduction in the number of parameters. This finding underscores the potential of parameter-efficient learning approaches.

Figure 4 reveals significant differences in precision across five categories among various models. FireCLIP achieves the highest precision, approaching 96.5%, in the “Field Burning and Wildfires” category, but underperforms in “Village Cooking Smoke”.

In contrast, CLIP exhibits generally weak performance, particularly showing extremely low precision in both “Field Burning and Wildfires” and “Illuminated by Light”, rendering it inadequate for classification purposes. The marked performance improvement of FireCLIP over CLIP highlights the advantages of prompt learning in fire scene classification tasks, suggesting that optimizing prompt learning strategies can effectively enhance the model’s feature extraction and generalization capabilities.

Fireformer and EfficientNetV2 demonstrate relatively stable performance, maintaining high precision across multiple categories, especially excelling in “Industrial Emissions” and “Village Cooking Smoke”, with precision exceeding 80%. ConvNeXt approaches optimal performance in “Field Burning and Wildfires” but shows slightly inferior results in “Village Cooking Smoke”. Overall, “Mist and Water Vapor” emerges as the most challenging category to classify, with all models performing poorly, whereas “Field Burning and Wildfires” is comparatively easier to classify. This performance is consistent with the data distribution.

3.4.2. Zero-Shot Transfer on WUISAD

Before exploring the model’s ability in zero-shot transfer, we first need to grasp the distinctions among the datasets to assess the generalization capabilities of different methods effectively.

To evaluate this, we utilized a pre-trained ResNet50 [44] model to extract features from multiple image datasets and calculated the cosine similarity and Euclidean distance between these datasets. Specifically, we input all images from each dataset into the pre-trained ResNet50 model to obtain feature vectors for each image. The average of these feature vectors was computed to represent the overall feature profile of each dataset.

Based on the extracted feature vectors, we quantified the similarity and divergence between datasets using cosine similarity and Euclidean distance. Cosine similarity measures the alignment of vector directions, reflecting semantic similarity, with values ranging from −1 to 1, where a higher value indicates greater similarity. In contrast, the Euclidean distance assesses absolute differences by computing the straight-line distance between vectors, with smaller distances indicating closer feature proximity.

Figure 5 provides information on similarities and differences between the datasets. The cosine similarity heatmap reveals that Datasets C and E exhibit the highest level of similarity (0.96), indicating their highly aligned feature distributions. Dataset A + B also demonstrates significant similarity with both Dataset C and Dataset E, with values exceeding 0.95. However, Dataset D shows relatively lower similarity with other datasets, particularly with Dataset C and Dataset E (values ranging between 0.84 and 0.9), suggesting that Dataset D possesses more distinct feature characteristics.

The Euclidean distance heatmap corroborates these findings, where shorter distances signify greater similarity. Dataset C and Dataset E exhibit the smallest Euclidean distance (9.7), reaffirming their close relationship in feature space. Dataset A + B also shows relatively low distances with Dataset C and Dataset E (10–12), supporting their shared feature alignment. Conversely, Dataset D is characterized by consistently higher distances (16–21) compared to other datasets, emphasizing its unique feature distribution.

Table 5. Zero-shot transfer performance of different methods on WUISAD.

	Source	Target
	Domain A + B	Domain C	Domain D	Domain E	Average
FireFormer	87.30	66.51	50.17	41.18	52.62
ConvNeXt	89.56	51.65	58.96	26.32	45.64
EfficientNetV2	81.63	47.81	52.32	26.90	42.34
FireCLIP	86.10	70.24	58.24	66.73	65.07

shows the performance of the baseline models and FireCLIP in zero-shot transfer across different domains (Domains C, D, and E). Zero-shot transfer is a method to test the model’s generalization ability, i.e., directly evaluating the model’s performance without any additional training on the target dataset.

FireCLIP exhibits the strongest transfer capability across all target domains (Domains C, D, and E), achieving accuracy rates of 70.24% and 66.73% in Domain C and Domain E, respectively. These results significantly outperform other methods, demonstrating its superior cross-domain generalization abilities. Moreover, FireCLIP achieves an average accuracy of 65.07%, which is substantially higher than FireFormer (52.62%), ConvNeXt (45.64%), and EfficientNetV2 (42.34%). This clearly highlights its overall advantage in zero-shot transfer learning.

FireFormer performs second to FireCLIP in the target domains, attaining an accuracy of 66.51% in Domain C, which indicates its robust cross-domain transfer capabilities. However, its performance in certain target domains, such as Domain D and Domain E, still falls short of FireCLIP, suggesting that its adaptability in zero-shot scenarios has room for improvement.

In contrast, ConvNeXt and EfficientNetV2 demonstrate weaker performance, especially in Domain E, where their accuracy rates are only 26.32% and 26.90%, respectively. This reveals a notable deficiency in cross-domain transfer capabilities, indicating that these two methods have limited generalization abilities when faced with tasks involving significant domain distribution differences.

3.4.3. Few-Shot Learning

Table 6. Binary classification accuracy (%) of each model after pre-training on the FASDD dataset under few-shot settings.

	FireCLIP	Fireformer	ConvNeXt	EfficientNetV2
4-shot	98.33	83.62	81.04	80.61
8-shot	98.43	84.51	85.67	80.83
16-shot	98.51	85.42	86.27	82.00
32-shot	98.56	87.23	88.60	82.69

demonstrates the binary classification performance of pre-trained models on the FASDD dataset under few-shot settings. FireCLIP achieves high accuracy (98.33–98.56% across 4–32 shots), significantly outperforming FireFormer (83.62–87.23%), ConvNeXt (81.04–88.60%), and EfficientNetV2 (80.61–82.69%), with performance gaps widening as sample size decreases. These results highlight FireCLIP’s superior few-shot adaptability through vision-language alignment while revealing conventional models’ limitations in low-data regimes.

The experimental results depicted in Figure 6 reveal significant performance disparities among various models in few-shot learning scenarios, particularly across different domains (Domain C, Domain D, and Domain E). Notably, FireCLIP consistently exhibits superior performance advantages. Under few-shot conditions, such as having only four or eight labeled samples per class, FireCLIP achieves markedly higher accuracy compared to other benchmark models, demonstrating robust generalization capabilities and thereby affirming its suitability for few-shot learning tasks. In contrast, FireFormer’s performance incrementally improves with an increasing number of samples, nearing FireCLIP’s performance when ample data is available (e.g., the ALL condition), yet it still shows a noticeable performance gap under limited sample conditions. Conversely, EfficientNetV2 and ConvNeXt display limited performance enhancements in few-shot settings, indicating a weaker adaptability to few-shot learning tasks.

Overall, FireCLIP’s superior performance in few-shot learning underscores its ability to maintain high classification accuracy despite limited labeled data, which is crucial for data-scarce applications such as forest fire detection. FireFormer, on the other hand, demonstrates strong learning capabilities when sufficient samples are available, highlighting its adaptability in resource-rich environments. In comparison, the constrained performance of EfficientNetV2 and ConvNeXt under few-shot conditions limits their potential applicability in related tasks.

The analysis presented above demonstrates the capability of various models in identifying different types of smoke. However, in practical application scenarios, it is crucial to enhance the accuracy of forest fire detection and reduce the rate of false alarms. Therefore, our subsequent focus will be on the precision and recall metrics of positive samples. By conducting an analysis of the precision and recall of positive samples, we can more accurately quantify the model’s performance in detecting forest fire events. The precision metric measures the proportion of actual forest fire events among all events predicted by the model as positive samples, which is of significant importance for assessing the accuracy of the model’s predictions. The recall, on the other hand, focuses on the proportion of real forest fire events that the model can identify out of all actual forest fire events, reflecting the comprehensiveness of the model in capturing forest fire events.

The experimental results presented in Figure 7 demonstrate significant fluctuations in both recall and precision across different methods, particularly when the number of labeled samples is limited (e.g., four or eight samples). For instance, FireCLIP shows a decrease in recall with increasing sample size in certain domains, potentially due to the lack of dedicated optimization for positive sample accuracy during model training.

In comparative analyses, FireCLIP exhibits relatively superior overall performance, maintaining high levels of recall and precision as the sample size increases. FireFormer’s performance is notably affected by the number of samples, yet it still demonstrates some capability for few-shot learning. In contrast, ConVNeXt and EfficientNetV2 show generally average performance, struggling with few-shot learning in Datasets C and D.

These findings suggest that maintaining high recall and precision for positive samples while enhancing overall accuracy remains a significant research challenge.

Considering the overall strong performance of FireFormer as a baseline, we will focus on comparing FireCLIP with FireFormer on WUISAD in the subsequent analysis.

In order to conduct a comprehensive evaluation of the models’ performance, we introduce the F1 score, which is the harmonic mean of precision and recall. This metric effectively integrates the considerations of both the precision and comprehensiveness of the models. Observations from Figure 8 indicate that FireCLIP outperforms FireFormer in the comprehensive prediction of positive samples, particularly in scenarios where the number of learnable samples is limited.

3.4.4. Ablation Experiments

Figure 9 presents an ablation study evaluating the impact of prompt length on FireCLIP’s performance during pre-training (Dataset A + B) and subsequent zero-shot transfer (average across Datasets C, D, E). The harmonic mean (H.M) quantifies the balance between source domain proficiency and cross-domain generalization capability.

The results reveal certain sensitivity to prompt length. While longer prompts generally enhance pre-training accuracy on the source domain, indicating improved adaptation to its specific characteristics, this benefit comes at a cost to generalization. Specifically, prompts exceeding an optimal length degrade zero-shot transfer performance across the target domains.

3.4.5. Visualization

To further demonstrate the robustness of FireCLIP in complex scenarios, more challenging cases are collected for evaluation, as illustrated in Figure 10. All hard cases are labeled as “Field Burning and Wildfires”, originating from three distinct domain. We select the FireCLIP and FireFormer models under a 32-shot setting for visualization analysis.

Domain C: The challenge in the Domain C hard case arises from the smoke generated in a wildland–urban interface (WUI) area, which can be confused with cooking smoke. The FireCLIP model primarily focuses on the wooded areas, which might be more typical of wildfires, whereas FireFormer directs its attention towards the buildings. This difference suggests that FireCLIP may better discriminate between different smoke sources in this environment, while FireFormer might be distracted by structures that could also be common in non-wildfire smoke scenarios.

Domain D: In this hard case, multiple distractions such as sunlight reflections on water surfaces present challenges. FireCLIP successfully concentrates on the correct smoke areas among the trees, indicating a robust capability to ignore misleading bright reflections. In contrast, FireFormer seems to be attracted by the sunlight, potentially reducing its effectiveness in accurately identifying smoke amidst environmental reflections.

Domain E: Similar to Domain C’s hard case, Domain E’s hard case also features obscured smoke sources near buildings. Although FireFormer correctly identifies the smoke’s location, its focus on the buildings can lead to misjudgments. FireCLIP, on the other hand, maintains a broader focus on the expansive wooded areas, potentially offering a more reliable detection of wildfire smoke when it originates from or spreads into forested regions.

Based on the aforementioned analysis, the FireCLIP model exhibits a relatively broad distribution of attention, particularly demonstrating heightened focus on forested areas. In contrast, the FireFormer model tends to be more easily attracted to the representational features of non-forested areas.

4. Discussion

FireCLIP demonstrates three advantages through its parameter-efficient architecture and multimodal prompt tuning strategy.

First, it achieves computational efficiency with only 3.55M trainable parameters—orders of magnitude fewer than FireFormer (197 M), ConvNeXt-L (198 M), and EfficientNetV2-L (119.5 M)—while maintaining high inference speeds (91 FPS), enabling scalable deployment in resource-constrained environments. Second, the framework exhibits exceptional zero-shot cross-domain generalization, outperforming the baseline models by 12.45% regarding the average accuracy (65.07% vs. FireFormer’s 52.62%) across geographically distinct regions (Table 5, effectively mitigating regional data imbalances through CLIP’s vision-language alignment. Third, FireCLIP excels in few-shot adaptation, achieving 98.33% accuracy in four-shot FASDD classification (Table 6—a 4.71–17.72% improvement over the other models—by leveraging task-specific prompts to extract discriminative smoke features from limited samples.

This unified combination of parameter efficiency, robust cross-regional adaptability, and data-scarce learning superiority addresses core challenges in wildfire detection, including pseudo-smoke interference and geographic distribution shifts, while minimizing computational overheads.

5. Conclusions

This study develops an efficient early warning system for forest fires, addressing a critical global environmental threat. We constructed a smoke-based dataset and introduced two strategies: a visual language framework to differentiate fire and non-fire samples and prompt tuning to enhance model generalization in data-scarce regions. Our model, FireCLIP, was evaluated using accuracy, precision, recall, and F1 scores, demonstrating superior performance in pre-training and zero-shot transfer, especially in new domain generalization. Compared to traditional fine-tuning, FireCLIP leverages CLIP’s pre-trained knowledge with prompt tuning for rapid adaptation with minimal parameter adjustments. In few-shot learning tasks, FireCLIP significantly outperformed FireFormer, showing improved accuracy and stable performance as the number of training samples increased. These results highlight FireCLIP’s high accuracy and reliability in predicting forest fires, enhancing the effectiveness of detection systems.

The forest fire detection early warning system proposed in this study has demonstrated strong performance both theoretically and practically; however, it also faces certain limitations and challenges:

• Vision-Aware Dynamic Prompting: While the current prompt templates primarily rely on static textual descriptions, incorporating fine-grained visual semantics specific to the content of the current input image presents a promising future direction. Static prompts inherently struggle to adapt to the diverse visual manifestations of smoke. Future works can focus on developing mechanisms to dynamically inject image-specific visual features or semantic descriptions into the prompt template generation process. Successfully integrating these dynamic, vision-guided elements with static contextual knowledge has the potential to significantly refine the model’s ability to recognize subtle smoke characteristics, leading to enhanced generalization and robustness across varied environmental scenes.
• Environmental and Seasonal Variability: In addition to the challenges of large and diverse negative samples and regional data imbalance discussed in the present work, it is important to consider the influence of environmental changes and seasonal variations. Forest fire detection models may be sensitive to factors such as varying vegetation, weather conditions, and seasonal changes, which could impact model performance. Future research should explore how to adapt and optimize detection systems for different environmental contexts and seasons to ensure consistent performance year-round.