Review of Modern Forest Fire Detection Techniques: Innovations in Image Processing and Deep Learning

Abstract

Fire detection and extinguishing systems are critical for safeguarding lives and minimizing property damage. These systems are especially vital in combating forest fires. In recent years, several forest fires have set records for their size, duration, and level of destruction. Traditional fire detection methods, such as smoke and heat sensors, have limitations, prompting the development of innovative approaches using advanced technologies. Utilizing image processing, computer vision, and deep learning algorithms, we can now detect fires with exceptional accuracy and respond promptly to mitigate their impact. In this article, we conduct a comprehensive review of articles from 2013 to 2023, exploring how these technologies are applied in fire detection and extinguishing. We delve into modern techniques enabling real-time analysis of the visual data captured by cameras or satellites, facilitating the detection of smoke, flames, and other fire-related cues. Furthermore, we explore the utilization of deep learning and machine learning in training intelligent algorithms to recognize fire patterns and features. Through a comprehensive examination of current research and development, this review aims to provide insights into the potential and future directions of fire detection and extinguishing using image processing, computer vision, and deep learning.

1. Introduction

Forests cover approximately 4 billion hectares of the world’s landmass, roughly equivalent to 30% of the total land [1]. The preservation of forests is essential for maintaining biodiversity on a global scale. Wildfires are destructive events that could adversely change the balance of our planet and threaten our future [2]. Wildfires have long-term devastating effects on ecosystems, such as destroying vegetation dynamics, greenhouse gas emissions, loss of wildlife habitat, and destruction of land covers. The early detection and rapid extinguishing of fires are crucial in minimizing the loss of life and property [3]. Traditional fire detection systems that rely on smoke or heat detectors suffer from low accuracy and long response times [4]. However, advancements in image processing (IP), computer vision (CV), and deep learning (DL) have opened up new possibilities for more effective and efficient fire detection and extinguishing systems [5]. These systems utilize cameras and sophisticated algorithms to analyze visual data in real-time, enabling early fire detection and efficient fire suppression strategies.

In most of the literature, researchers have mainly posed their problem under the paradigm of fire detection [6,7,8]. But some researchers have also explored different aspects of the phenomenon of combustion i.e., smoke [9,10], flame [11], and fire [12], with the intent to effectively determine the threats due to fire. In summary, fire is the overall phenomenon of combustion involving the rapid oxidation of a fuel source, while flame represents the visible, gaseous part of a fire that emits light and heat. Smoke, on the other hand, is the collection of particles and gases released during a fire, which can be toxic and pose health hazards [13]. In this paper, we review the automatic fire, flame, and smoke detection for the last eleven years, i.e., from 2013–2023, using deep learning and image processing.

Image processing techniques enable the extraction of relevant features from images or video streams that are captured by cameras [14]. This includes analyzing color, texture, and spatial information to identify potentially fire-related patterns [15]. By applying algorithms such as edge detection, segmentation, and object recognition, fire can be detected and differentiated from non-fire elements with a high degree of accuracy [16,17].

Computer vision can play a crucial role in early fire detection by utilizing image and video processing techniques to analyze visual data and identify signs of fire [18]. CV algorithms can identify patterns based on features such as color, shape, and motion [19,20]. CV with thermal imaging technology can detect fires based on temperature variations [21,22]. It is important to note that CV conjugated with other fire safety measures, such as smoke detectors, heat sensors, and human intervention, enhances early fire detection. DL combined with CV can also effectively recognize various fire characteristics, including flames, smoke patterns, and heat signatures [23]. It enables more precise and reliable fire detection, even in challenging environments with variable lighting conditions or occlusions.

Deep learning, a subset of machine learning (ML), has revolutionized the field of CV by enabling the training of highly complex and accurate models [24]. Deep learning models, such as convolutional neural networks (CNNs), can be trained on vast amounts of labeled fire-related images and videos, learning to automatically extract relevant features and classify fire instances with remarkable precision [25,26]. These models can continuously improve their performance through iterative training, enhancing their ability to detect fires and reducing false alarms [27].

This work provides a systematic review of the most representative fire and/or smoke detection and extinguishing systems, highlighting the potential of image processing, computer vision, and deep learning. Based on three types of inputs, i.e., camera images, videos, and satellite images, the widely used methods for identifying active fire, flame, and smoke are discussed. As research and development continue to advance these technologies, future fire extinguishing systems promise to provide robust protection against the devastating effects of fires, ultimately saving lives and minimizing property damage.

The remainder of this paper is structured as follows: Section 2 presents the search strategy and selection criteria. Section 3 details the broadly defined classes for fire and smoke detection. Section 4 presents an analysis of the selected topic areas, discussing representative publications from each area in detail. In Section 5, we provide the discussion related to the factors critical for forest fire, followed by the recommendations for future research in Section 6. Lastly, Section 7 concludes this study with some concluding thoughts.

2. Methodology: Search Strategy and Selection Criteria

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [28] framework defined the methodology for this systematic review. PRISMA provides a standardized approach for conducting and reporting systematic reviews, ensuring that all relevant studies are identified and assessed comprehensively and transparently. This review aims to understand the approaches used to detect or extinguish forest fires. The required data for this systematic review were gathered from two renowned sources, Web of Science^™ and IEEE Xplore^®, and the review was limited to peer-reviewed journal articles published from 2013 to 2023. Web of Science^™ is a research database that offers a wide range of scholarly articles across many disciplines. It includes citation indexing, which helps track the impact of research. IEEE Xplore^® is a digital library focused on electrical engineering, electronics, computer science, and other related fields. It provides access to technical literature like journal articles, conference proceedings, and technical standards. We used the EndNote 20.6 reference manager, a software tool by Clarivate, to organize and manage the references collected during the review process. EndNote helped us to classify the references, filter relevant studies, and screen for duplicates, as well as ensure a comprehensive and systematic review of the literature. This tool is widely used in academic research to streamline the process of citation management and bibliography creation. “Fire Detection” was used in conjunction with “Computer Vision”, “Machine Learning”, “Image Processing”, and “Deep Learning” to define the primary search string. To identify the applications of fire detection, “Fire Extinguishing” conjugated with “UAV” and “UGV” was used to define the secondary search string. The pictorial view of the selected areas of the research along with their distribution is depicted in Figure 1.

Figure 1. Selected areas for research.

Figure 2 illustrates the PRISMA framework used to identify and select the most relevant literature. As a result of the research conducted using the primary keywords, 1872 records in Web of Science^™ and 288 records in IEEE Xplore^® were retrieved. Data from both sources were merged and after duplicate removal, 1823 records were left. By excluding all records published before 2013 and after 2023, and by applying the search string (“Forest Fire” $\left|\right|$ “Wildfire”) (“detection” $\left|\right|$ “recognition” $\left|\right|$ “extinguish”) in the abstract, title, and keyword fields, only 270 were retained. Another screening was applied to obtain the most relevant data aligned with our interest and by excluding publications for which the full text was not accessible, a total of 155 journal papers from the most relevant journals were retained for detailed review.

Figure 2. PRISMA framework.

To analyze these publications, Figure 3 illustrates the number of journal publications from 2013–2023. The increasing trend after 2018 is an indicator of growing interest in this area of study. The top five journals publishing the most papers on this topic are Fire Technology (9), Forests (14), IEEE Access (9), Remote Sensing (21), and Sensors (13). These journals account for almost 43% of all publications.

Figure 3. Distribution of the number of publications over the period of 2013 to 2023.

3. Research Topics

While conducting our literature search, we tried to cover all aspects contributing to the overall topic. Though these can be considered distinct research topics, from the perspective of deep learning, they play their part mutually.

• Image Processing: Research that focuses on fire detection based on the features extracted after processing the image [29,30].
• Computer Vision: Research focusing on the algorithms to understand and interpret the visual data to identify fire [31].
• Deep Learning: Research associated with the models that can continuously enhance their ability to detect fires [32].

Based on the literature search, four main groups were formulated to classify the publication results. This classification is mainly based on the research topic, theme, title, practical implication, and keywords. Each publication in our search fell broadly into one of these categories:

• Fire: Research that addresses the methods capable of identifying the forest fire in real-time or based on datasets [33,34].
• Smoke: Research focusing on the methods to identify smoke with its different color variations [35,36].
• Fire and Flame: Research associated with the methods that can identify fire and flame [37].
• Fire and Smoke: Research that explores the methods focusing on the accurate determination of fire and smoke [38].
  Another category has been introduced that is a part of the above-defined categories in the field, but with application orientation, with the help of robots.
• Applications: Research that addresses a robot’s ability not only to detect fire but also to extinguish it [39,40,41].

4. Analysis

The distribution of various publications in selected categories is illustrated in Figure 4. From the defined categories, fire detection was the most dominant class containing 68 (44%) of the 155 total publications, followed by smoke detection with 33 (21%), fire and smoke with 23 (15%), applications with 18 (12%), and fire and flame with 13 (8%). The data highlight that fire detection and monitoring are foundational areas in the field, while practical applications for fire extinguishing, particularly those involving unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs), remain less developed. Only seven articles focused on UGVs and 11 on UAVs for fire extinguishing, indicating that on-filed utilization in this area is still in its early stages.

Figure 4. Publications in selected categories.

Deep learning has been successfully applied to fire, flame, and smoke detection tasks, where its ability has been utilized to learn complex patterns and features from large amounts of data [42,43]. The primary task in fire detection is dataset collection, which consists of a large dataset of images or videos containing both fire and non-fire scenes [44]. The collected data need to be preprocessed to ensure consistency and quality. This may involve resizing images, normalizing pixel values, removing noise, and augmenting the dataset by applying transformations like rotation, scaling, or flipping [45]. Afterward, a deep learning model needs to be designed and trained to perform fire, smoke, or flame detection. CNNs are commonly used for this purpose due to their effectiveness in image-processing tasks [46]. The architecture can be customized based on the specific requirements and complexity of the detection task [47].

For all publications, we extracted some key information such as dataset, data type, method, objective, and achievement. One or two representative publications were picked from each category based on the annual citation count (ACC). The ACC is a metric that indicates the average number of citations per year since publication. The citation count was retrieved from the Web of Science^™ till July 2024. To qualify for the representative publication, each publication’s ACC should have a positive standard deviation, Std (ACC).

4.1. Fire

It is important to note that deep learning models for fire detection rely heavily on the quality and diversity of the training data. Obtaining a comprehensive and representative dataset is crucial for achieving accurate and robust fire detection performance. Past research efforts related to fire detection are listed in Table 1 in terms of the dataset, method, objectives, and achievements.

Table 1. List of the past work related to fire detection.

Ref	Dataset	Data Type	Method	Objective	Achievement
[48]	47,992 images	Images	Transfer learning	Achieving early prevention and control of large-scale forest fires.	Recognition accuracy of 79.48% through FTResNet50 model.
[49]	2976 images	Images	YOLOv5 and EfficientDet	Overcoming the shortcomings of manual feature extraction and achieving higher accuracy in forest fire recognition by weighted fusion.	The average accuracy of the proposed model for forest fire identification reached 87%.
[50]	11 videos	Videos	YCbCr and correlation coefficient	Achieving efficient forest fire detection using rule-based multi-color space and a correlation coefficient.	Achieved 95.87% and 97.89% of F-score and accuracy on fire detection.
[51]	11,456 images	Images	SqueezeNet	Identifying the existence of fire by first segmenting all fire-like areas and then processing through the classification module.	Attained 93% accuracy.
[52]	2100 images	Images	CNN	Attempting to extract and classify image features for fire recognition based on CNN.	Achieved a classification accuracy of around 95%.
[53]	* data obtained from USGS website	Satellite images	SVM	Performing forest fire detection on LANDSAT images using SVM.	Obtained 99.21% accuracy and a high precision of 98.41% on fire detection.
[54]	12,000 frames	Thermal images	Automatic gain control algorithm	Utilizing thermal infrared sensing for near real-time, data-driven fire detection and monitoring.	The proposed approach achieved better situation awareness when compared to existing methods.
[55]	37 images	Satellite images	Simple linear iterative clustering	Building an unsupervised change detection framework that uses post-fire VHR images with prefire PS data to facilitate the assessment of wildfire damage.	Achieved an overall accuracy of over 99% on wildfire damage assessments.
[56]	500 images	Images	YCbCr color space and CNN	Introducing conventional image processing techniques, CNNs, and an adaptive pooling approach.	Achieved an accuracy of 90.7% on fire detection.
[57]	52 images	Images	MWIR	Detecting forest fires by middle infrared channel measurement.	Achieved 77.63% accuracy on fire detection.
[58]	*	Images	Horn and Schunck optical flow	Performing aerial images-based forest FD for firefighting using optical remote-sensing techniques.	Experimental results have verified that the proposed forest fire detection method can achieve good performance.
[59]	175 videos	Videos	SVM	Performing multi-feature analysis in YUV color space for early forest FD.	Attained an average detection rate of 96.29%.
[60]	VIIRS	Satellite images	FILDA	Developing FILDA that characterizes fire pixels based on both visible light and IR signatures at night.	Compared to the existing algorithms, the proposed algorithm produced a much more accurate detection of fire.
[61]	13 images	Images	Spatio-temporal model	Developing a spatio-temporal model for forest FD using HJ-IRS satellite data	Achieved 94.45% detection rate on fire detection.
[62]	5 images	Images	GMM	Building an early detection system of forest fire smoke signatures using GMM.	The developed system detected fire in all of the test videos in less than 2 min.
[63]	3320 images	Images	YOLOv5	Performing small-target forest fire detection.	Achieved an 82.1 mAP@0.5 in forest fire detection and a 70.3 mAP@0.5-S in small-target forest fire detection.
[64]	22 tiles of Landsat-8 images	Satellite images	Deep CNN	Determining the starting point of the fire for the early detection of forest fires.	Achieved a 97.35% overall accuracy under different scenarios.
[65]	11,681 images	Images	FCOS	Detecting forest fires in real-time and providing firefighting assistance.	Attained 89.34% accuracy in forest fire detection.
[66]	6595 images	Images	MTL	Solving the problems of poor small-target recognition and many missed and false detections in complex forest scenes.	Achieved 98.3% accuracy through segmentation and classification.
[67]	8000 images	Images	R-CNN	Classifying video frames as two classes (fire, no-fire) according to the presence or absence of fire and the segmentation method used for incipient forest-fire detection and segmentation.	An accuracy of 93.65% and a precision of 91.85% were achieved on forest-fire detection and segmentation.
[68]	*	Images	Non-sub-sampling contourlet transform and visual saliency	Building a machine vision-based network monitoring system for solar-blind ultraviolet signals.	It was claimed that the fusion results of the proposed method had higher clarity and contrast, and retained more image features.
[69]	81,810 images	Images	R-CNN, Bayesian network, and LSTM	Improving fire detection accuracy when compared with other video-based methods.	Achieved an accuracy of 97.68% for affected areas.
[70]	500 images	RGB and NIR image	Vision transformer	Achieving early detection and segmentation to predict their spread and help with firefighting.	Obtained a 97.7% F1-score on wildfire segmentation.
[71]	2000 images	Images	Artificial bee colony algorithm-based color space	Detecting forest fires using color space.	Obtained an evaluated mean Jaccard index value of 0.76 and a mean Dice index value of 0.85.
[72]	4000 images	Images	Deep CNN	Detecting fire as early as possible.	Achieved a 94.6% F-score fire detection rate.
[73]	48,010 images	Images	CNN and vision transformers	Detecting wildfire at early stages.	Obtained a 85.12% accuracy on wildfire classification and a 99.9% F1-score on semantic segmentation.
[74]	37,016 images	Satellite images	CNN	Building automated an active fire detection framework using Sentinel-2 imagery.	Obtained an average IoU higher than 70% on active fire detection.
[75]	38,897 images	Satellite images	CNN	Accurately detecting the fire-affected areas from satellite imagery.	Achieved a 92% detection rate under cloud-free weather conditions.
[76]	8194 images	Satellite images	CNN	Performing active fire detection using deep learning techniques.	Achieved a precision of 87.2% and a recall of 92.4% on active fire detection.
[77]	10,000 images	Images	RNN, LSTM, and GRU	Performing early detection of forest fires with higher accuracy.	An accuracy of 99.89% and a loss function value of 0.0088 were achieved on fire detection.
[78]	*	Satellite images	GRU network	Building an early fire detection system.	Performed GRU-based detection of the wildfire earlier than the VIIRS active fire products in most of the study area.
[79]	5469 images	Satellite images	CNN	Building an accurate monitoring system for wildfires.	Achieved an accuracy of 99.9% on fire detection.
[80]	10,581 images	Images	EfficientDet and YOLOv5	Detecting forest fires in different scenarios by an ensemble learning method.	Obtained 99.6% accuracy on fire detection.
[81]	4000 images	Images	CNN	Introducing an additive neural network for forest fire detection.	Attained 96% accuracy on fire detection.
[82]	1500 images	Images	DCNN	Performing saliency detection and DL-based wildfire identification in UAV imagery.	Achieved an overall accuracy of 98% on fire classification.
[83]	6137 images	Images	CNN	Building a system that can spot wildfire in real-time with high accuracy.	Achieved detection precision of 98% for fire detection.
[84]	2425 images	Images	GMM-EM	Detecting fire based on combining color-motion-shape features with machine learning.	A TPR of 89.97% and an FNR of 10.03% were achieved for detection.
[85]	*	Images	CEP	Performing real-time wildfire detection with semantic explanations.	Through experimental results based on four real datasets and one synthetic dataset, the supremacy of the proposed method was established.
[86]	12 images and 7 videos	Images and videos	kNN	Performing pixel-level automatic annotation for forest fire images.	Achieved a higher fire detection rate and a lower false alarm rate in comparison to existing algorithms.
[87]	39,375 frames	Videos	ANN	Developing a dataset of aerial images of fire and performing fire detection and segmentation on this dataset.	Achieved a precision of 92% and a recall of 84% for detection.
[88]	2000 images	Images	CNN and SVM	Developing a robust algorithm to deal with the problems of a complex background, the weak generalization ability of image recognition, and low accuracy.	Accomplished fire detection with a recognition rate of 97.6%, a false alarm rate of 1.4%, and a missed alarm rate of 1%.
[89]	2 Landsat-7 images	Satellite images	ELM	Utilizing an adaptive ensemble of ELMs for the classification of RS images into change/no change classes.	Achieved an accuracy of 90.5% in detecting the change.
[90]	30 images	Videos and Images	SVM	Identifying fires and providing fire warnings yielding excellent noise suppression and promotion.	Obtained a 97% TPR on classification.
[91]	8500 images	Images	Data fusion	Detecting smoke from fires, usually within 15 min of ignition.	Achieved an accuracy of 91% on the test set and an F-1 score of 89%.
[92]	WSN	Transmission data	AAPF	Utilizing auto-organization and adaptive frame periods for forest fire detection.	Developed a comprehensive model to evaluate the communication delay and energy consumption.
[93]	20,250 pixels	Satellite images	Random forest	Building a three-step forest fire detection algorithm by using Himawari-8 geostationary satellite data.	Achieved an overall accuracy of 99.16%, a POD of 93.08%, and a POFD of 0.07%.
[94]	1194 images	Images	Multi-channel CNN	Performing fire detection using multichannel CNN.	Obtained 98% or more classification accuracy and claimed improvement by 2% than the traditional feature-based methods.
[95]	7690 images	Images	DCNN and BPNN	Developing an improved DCNN model for forest fire risk prediction. Implementing the BPNN fire algorithm to calculate video image processing speed and delay rate.	Achieved an 84.37% accuracy in real-time forest fire recognition.
[96]	*	Images	DeepLabV3+	Presenting Defog DeepLabV3+ for collaborative defogging and precise flame segmentation. Proposing DARA to enhance flame-related feature extraction.	Achieved a 94.26% accuracy, 94.04% recall, and 89.51% mIoU.
[97]	1452 images	Images	Transfer learning	Exploring several CNN models, applying transfer learning, using SVM and RF for detection, and using train/test networks with random and ImageNet weights on a forest fire dataset.	Achieved a 99.32% accuracy.
[98]	14,094 images	Images	FuF-Det (encoder–decoder transformer)	Designing AAFRM to preserve positional features. Constructing RECAB to retain fine-grained fire point details. Introducing CA in the detection head to improve localization accuracy	Achieve an AP@0.5 of 86.52% and a fire spot detection rate of 78.69%.
[99]	3000 images	Images	YOLOv5	Integrating the transformer module into YOLOv5’s feature extraction network. Inserting the CA mechanism before the YOLOv5 head. Using the ASFF in the model’s head to enhance multi-scale feature fusion.	Achieved an mAP@0.5 of 84.56%.
[100]	1900 images	Images	Ensemble learning	Proposing a stacking ensemble model. Using pre-trained models as base learners for feature extraction and initial classification, followed by a Bi-LSTM network as a meta-learner for final classification.	Achieved 97.37%, 95.79%, and 95.79% accuracy with hold-out validation, five-fold cross-validation, and tenfold cross-validation.
[101]	5250 infrared images	Images	YOLOv5s	Proposing FFDSM based on YOLOv5s-seg and incorporating ECA and SPPFCSPC modules to enhance fire detection accuracy and feature extraction.	Achieved an mAP@0.5 of 0.907.
[102]	204,300 images	Images	Deep ensemble learning	Presenting a deep ensemble neural network model using Faster R-CNN, RetinaNet, YOLOv2, and YOLOv3.	The proposed approach significantly improved detection accuracy for potential fire incidents in the input data.
[103]	1900 images	Images	CNN	Proposing a forest fire detection method using CNN architecture. Employing separable convolution layers for immediate fire detection, reducing computational resources, and enabling real-time applications.	Achieved an accuracy of a 97.63% and an F1-score of 98.00%.
[104]	51,906 images	Images	Ensemble learning	Proposing CT-Fire by combining deep CNN RegNetY and vision transformer EfficientFormer v2 to detect forest fires in ground and aerial images.	Attained accuracy rates of 99.62% for ground images and 87.77% for aerial images.
[105]	348,600 images	Images	Detectron2	Detecting forest fires using different deep-learning models. Preparing a dataset. Comparing the proposed method with existing ones. Implementing it on Raspberry Pi for CPU and GPU utilization.	Achieved a precision of 99.3%.
[106]	1900 images	Images	FL and PSO	Integrating PSO with FL to optimize communication time. Developing a CNN model incorporating FL and PSO to set basic parameters based on local client data. Enhancing FL performance and reducing latency in disaster response.	Achieved a prediction accuracy of 94.47%.
[107]	* data obtained from Landsat-8	Satellite images	U-Net	Introducing FU-NetCastV2. Collecting historic GeoMac fire perimeters, elevation, and satellite maps. Retrieving 24-h weather data. Implementing and optimizing U-Nets. Generating a burned area map.	Achieved an accuracy rate of 94.6% and an AUC score of 97.7%.
[108]	5060 images and 14,320 s audio	Images and audio	CNN	Proposing a VSU prototype with embedded ML algorithms for timely forest fire detection. Collecting and utilizing two datasets and audio and picture data for training the ML algorithm.	Achieved a 96.15% accuracy.
[109]	210 images	360-degree images	Multi-scale vision transformer	Introducing a FIRE-mDT model combining ResNet-50 and multiscale deformable transformer for early fire detection, location, and propagation estimation. Creating a dataset from real fire events in Seich Sou Forest.	Achieved an F-score of 91.6%.
[110]	55,746 images	Images	ANN and CNN	Proposing EdgeFireSmoke++, based on EdgeFireSmoke, using ANN in the first level and CNN in the second level.	Achieved over 95% accuracy.
[111]	23,982 images	Images	FireYOLO and Real-ESRGAN	Proposing a two-step recognition method combining FireYOLO and ESRGAN Net. Using GhostNet with dynamic convolution in FireYOLO’s backbone to eliminate redundant features. Enhance suspected small fire images with Real-ESRGAN before re-identifying them with FireYOLO.	Achieved a 94.22% average precision when implemented on embedded devices.
[112]	48 videos	Videos	Vision transformers (ViTs) and CNNs	Proposing FFS-UNet, a spatio-temporal architecture combining a transformer with a modified lightweight UNet. Extracting keyframe and reference frames using three encoder paths for feature fusion, and then using a transformer for deep temporal-feature extraction. Finally, segmenting the fire using shallow keyframe features with skip connections in the decoder path.	Achieved a 95.1% F1-score and 86.8% IoU on the UAV-collected videos, as well as a 91.4% F1-score and 84.8% IoU on the Corsican Fire dataset.
[113]	3800 images	Images	CNN	Proposing FireXnet, a lightweight model for wildfire detection that is suitable for resource-constrained devices. Incorporating SHAP to make the model’s decisions interpretable. Compare FireXnet’s performance against five pre-trained models.	Achieved an accuracy of 98.42%.
[114]	4674 images	Images	YOLOv5	Utilizing four detection heads in FireDetn. Integrating transformer encoder blocks with multi-head attention. Fusing the spatial pyramid pooling fast structure in detecting multi-scale flame objects at a lower computational cost.	Achieved an AP50 of 82.6%.
[115]	2 active fire products and 1 burned area product	Satellite images	Temporal patterns and kernel density estimation (KDE)	Comparing various MODIS fire products with ground wildfire investigation records in southwest China to identify differences in the spatio-temporal patterns of regional wildfires detected and exploring the influence of instantaneous and local environmental factors on MODIS wildfire detection probability.	Detected at least twice as many wildfire events as that in the ground records.

* Information not available.

• Representative Publications:

The annual citation count for all the papers listed in this category was calculated and is illustrated in Figure 5. The paper entitled “A Forest Fire Detection System Based on Ensemble Learning” was selected from this category as a representative publication, published in 2021, due to its highest ACC score [80]. In this work, the authors developed a forest fire detection system based on ensemble learning. First, two individual learners YOLOv5 and EfficientNet, were integrated to accomplish fire detection. Secondly, another individual learner, EfficientNet, was introduced for learning global information to avoid false positives. The used dataset contains 2976 forest fire images and 7605 non-fire images. Sufficient training sets enabled EfficientNet to show a good discriminability between fire objects and fire-like objects, with 99.6% accuracy on 476 fire images and a 99.7% accuracy on 676 fire-like images.

Figure 5. ACC and its standard deviation (- - -) for fire.

4.2. Smoke

Deep learning models learn to extract relevant features from input data automatically. During training, the model can learn discriminative features from smoke images that are independent of color. By focusing on shape, texture, and spatial patterns rather than color-specific cues, the model becomes less sensitive to color variations and can detect smoke effectively. Table 2 highlights the research focused on smoke detection.

Table 2. List of the past works related to smoke detection.

Ref	Dataset	Data Type	Method	Objective	Achievement
[116]	6 videos	Videos	Fusion deep network	Enhancing the detection accuracy of smoke objects through video sequences.	Achieved a 94.57% accuracy on smoke detection.
[117]	2977 images	Images	GIS and augmented reality	Improving the detection range and the rate of correct detection and reducing false alarm rates.	Managed to reduce the false alarm rate to 0.001.
[118]	6225 images	Images	Class activation map and ResNet-50	Building a class activation map-based data augmentation system for smoke scene detection.	Achieved the best accuracy of 94.95%.
[119]	90 videos	Videos	3D convolution-based encoder/decoder network	Building a 3D convolution-based encoder–decoder network architecture for video semantic segmentation.	Achieved a 99.31% accuracy on wildfire smoke segmentation.
[120]	90 videos	Videos	CNN	Building a 3D fully convolutional network for segmenting smoke regions.	Achieved a 0.7618 mAP on smoke detection.
[121]	50,000 images	Images	CNN	Performing real-time forest smoke detection using hand-designed features and DL.	The detection model achieved 97.124% accuracy on the test set.
[122]	38 smoke videos and 20 non-smoke videos	Videos	CNN	Detection of wildfire smoke based on faster RCNN and 3D CNNN.	Achieved a 95.23% accuracy on smoke detection.
[123]	22 videos	Videos	Vibe algorithm	Detecting forest fire smoke based on a visual smoke root and diffusion model.	Achieved an accuracy higher than 90% on smoke detection.
[124]	37,712 images	Images	Stereo vision triangulation	Achieving wildfire smoke detection using stereo vision.	Obtained results with an over 0.95 TPR on smoke detection.
[125]	11 videos	Videos	Saliency maps	Building a saliency-based method for early smoke detection through video sequences.	Achieved an average smoke segmentation precision of 93.0% and a precision as high as 99.0% for forest fires.
[126]	3225 images	Images	TECNN	Classification of smoke-like scenes in remote sensing images.	Obtained a 98.39% accuracy on smoke classification.
[127]	3645 images	Images	R-CNN	Detecting smoke columns that are visible below or above the horizon.	Produced an F1-score of 80%, a G-mean of 80%, and a detection rate of 90%.
[128]	1073 videos	Videos	DETR	Developing an open-source transformer-supercharged benchmark for fine-grained wildfire smoke detection.	Detected 97.9% of the fires in the incipient stage and 80% within 5 min from the start.
[129]	240 videos	Videos	CNN	Developing an intelligent smoke detection algorithm for wildfire monitoring cameras.	The overall fire risk of the test region is reduced to just 36.28% of its original value.
[130]	460 custom images	Images	GLCM, LBP, an ANN	Achieving a forest fire flame and smoke detection from UAV-captured images using fire-specific color features and multi-color space local binary patterns.	Achieved an F1-score of 90% for smoke detection.
[131]	4595 images	Images	CNN	Detecting wildfire smoke images based on a densely dilated CNN.	Achieved a 99.2% accuracy on smoke detection.
[132]	2000 images	Images	LSTM	Utilizing enhanced bidirectional LSTM for early forest fire smoke recognition.	Obtained an accuracy of 97.8% on smoke detection.
[133]	240 videos	Videos	HDLBP, CoLBP, and ELM	Achieving a lesser rate of incorrect alarms by identifying the smoke and examining its distinctive texture attributes.	Results obtained with 95% F1-score on fire detection.
[134]	500 images	Images	Multi-spectral fusion algorithm	Developing a wildfire image dataset and performing analysis on that dataset.	A tool was built for researchers and professionals through which they can access the dataset and also contribute.
[135]	6500 images	Images	YOLOv7	Collecting forest fire smoke photos, utilizing YOLOv7, incorporating CBAM attention mechanism, and applying SPPF+ and BiFPN modules to focus on small-scale forest fire smoke.	Achieved an $A{P}_{50}$ of 86.4% and an $A{P}_L$ of 91.5%
[136]	2554 images	Images	YOLOv5 and transfer learning	Improving YOLOv5s using K-means++ for anchor box clustering, adding a prediction head for small-scale smoke detection, replacing the backbone with $PConv$ for efficiency, and incorporating coordinate attention for region focus.	Achieved an $A{P}_{50}$ of 96% and an $A{P}_{50:95}$ of 57.3%.
[137]	10,250 images	Images	Deformable DETER	Proposing an improved deformable DETR model with MCCL and DPPM modules to enhance low-contrast smoke detection. Implementing an iterative bounding box combination method for precise localization and bounding of semi-transparent smoke.	Achieved an improvement of mAP (mean average precision) of 4.2% and anAPS (AP for small objects) of 5.1%.
[138]	6000 images	Images	YOLOv8	Incorporating WIoUv3 into a bounding box regression loss, integrating BiFormer into the backbone network, and using GSConv as a substitute for conventional convolution within the neck layer.	Achieved an average precision (AP) of 79.4%, an average precision small (APS) of 71.3%, and an average precision large (APL) of 92.6%.
[139]	5311 images	Images	YOLOv7	Proposing a lightweight model. Using GSConv in the neck layer, embedding multilayer coordinate attention in the backbone, utilizing the CARAFE up-sampling operator, and applying the SIoU loss function.	Achieved an accuracy of 80.2%.
[140]	1664 images	Images	Transformer	Proposing the FireFormer model. Using a shifted window self-attention module to extract patch similarities in images. Applying GradCAM to analyze and visualize the contribution of image patches.	Achieved an OA, Recall, and F1-score of 82.21%, 86.635%, and 74.68%, respectively.
[141]	35,328 images	Images	EfficientDet	Detecting distant smoke plumes several kilometers away using EfficientDet.	Achieved an 80.4% true detection rate and a 1.13% false-positive rate.
[142]	43,060 images	Images	LMINet	Proposing a deformable convolution module. Introducing a multi-direction feature interaction module. Implementing an adversarial learning-based loss term.	Achieved a mIoU and pixel-level F-measure of 79.31% and 84.61%, respectively.
[143]	77,910 images	Images	PSNet	Utilizing non-binary pixel-level supervision to guide model training. Introducing DDAM to distinguish smoke and smoke-like targets, AFSM to enhance smoke-relevant features, and MCAM for enhanced feature representation.	Achieved a detection rate of 96.95%.
[144]	614 images	Images	CNN	Optimizing a CNN model. Training MobileNet to classify satellite images using a cloud-based development studio and transfer learning. Assessing the effects of input image resolution, depth multiplier, dense layer neurons, and dropout rate.	Achieved a 95% accuracy.
[145]	6225 images	Satellite images	CNN	Introducing SmokeNet, a new model using spatial and channel-wise attention for smoke scene detection, including a unique bottleneck gating mechanism for spatial attention.	Achieved a 92.75% accuracy.
[146]	975 images	Satellite images	FCN	Presenting a deep FCN for a near-real-time prediction of fire smoke in satellite imagery.	Achieved a 99.5% classification accuracy.
[147]	24,217 images	Images	Deep multi-scale CNN	Designing a multi-scale basic block with parallel convolutional layers of different kernel sizes and merging outputs via addition to reduce dimension. Proposing a deep multi-scale CNN using a cascade of these basic blocks.	Achieved a 95% accuracy.
[148]	20,000 images	Images	DCNN	Presenting a smoke detection method using a dual DCNN. The first framework extracts image-based features like smoke color, texture, and edge detection. The second framework extracts motion-based features, such as moving, growing, and rising smoke regions.	Achieved an average accuracy of 97.49%.

• Representative Publications:

The ACC score for all the publications falling in the category was determined and is illustrated in Figure 6. Based on the plot, the two best performers were chosen from this category. A notable publication [143] titled ‘Learning Discriminative Feature Representation with Pixel-Level Supervision for Forest Smoke Recognition,’ focuses on forest smoke recognition through using a Pixel-Level Supervision Neural Network. The research employed non-binary pixel-level supervision to enhance model training, introducing a dataset of 77,910 images. To improve the accuracy of smoke detection, the study integrated the Detail-Difference-Aware Module to differentiate between smoke and smoke-like targets, the Attention-based Feature Separation Module to amplify smoke-relevant features, and the Multi-Connection Aggregation Method to enhance feature representation. The proposed model achieved a remarkable detection rate of 96.95%.

Figure 6. ACC and its standard deviation (- - -) for smoke.

The second representative publication, titled ‘SmokeNet: Satellite Smoke Scene Detection Using Convolutional Neural Network with Spatial and Channel-Wise Attention’ [145] and published in 2019, aimed to detect wildfire smoke using a large-scale satellite imagery dataset. It proposed a new CNN model, SmokeNet, which incorporates spatial and channel-wise attention for enhanced feature representation. The USTC_SmokeRS dataset, consisting of 6225 images across six classes (cloud, dust, haze, land, seaside, and smoke), served as the benchmark. The SmokeNet model achieved the best accuracy rate of 92.75% and a Kappa coefficient of 0.9130, outperforming other state-of-the-art models.

4.3. Fire and Flame

Deep learning models can integrate multiple data sources to improve fire and flame detection. In addition to visual data, other sources such as thermal imaging, infrared sensors, or gas sensors can be used to provide complementary information. By fusing these multi-modal inputs, the model can enhance its ability to detect fire and flame accurately, even in challenging conditions. The existing work related to fire and flame detection is presented in Table 3.

Table 3. List of the past works related to fire and flame detection.

Ref	Dataset	Data Type	Method	Objective	Achievement
[149]	338 images	Images	FSCN and ISSA	Improving the accuracy of fire recognition with a fast stochastic configuration network.	Achieved a 94.87% accuracy on fire detection.
[150]	5 videos	Videos	Unsupervised method	Achieving the early detection of wildfires and flames from still images by a new unsupervised method based on RGB color space.	Achieved a 93% accuracy on flame detection.
[151]	14 videos	Videos	K-SVD	Detecting wildfire flame using videos from pixel to semantic levels.	Obtained a 94.1% accuracy on flame detection.
[152]	85 videos	Videos	ELM	Performing a static and dynamic texture analysis of flame in forest fire detection.	Attained an average detection rate of 95.65%.
[153]	101 images	Images	SVM	Devising a new fire detection and identification method using a visual attention mechanism.	Accomplished an accuracy of 82% for flame recognition.
[154]	51,998 images and 6 videos	Images & Videos	YOLOv5n	Applying YOLOv5 to detect forest fires from images captured by UAV and analyzing the flame detection performance of YOLOv5.	Achieved a detection speed of 1.4 ms/frame and an average accuracy of 91.4%.
[155]	1900 images	Images	CNN	Proposing wildfire image classification with Reduce-VGGnet and region detection using an optimized CNN, combining spatial and temporal features.	Achieved an accuracy of 97.35%.
[156]	2603 images	Images	ADE-Net	Introducing a dual-encoding path with semantic and spatial units, integrating AFM, using an MAF module, proposing an AGE module, and finally employing a GCF module.	Achieved a 90.69% and 80.25% Dice coefficient, as well as a 91.42% and 83.80% mIOU, on the FLAME and Fire_Seg datasets, respectively.
[157]	20 videos	Videos	Optic flow	Proposing the following four-step algorithm: preprocessing input data, detecting flame regions using HSV color space, modeling motion information with optimal mass transport optical flow vectors, and measuring the area of detected regions.	Achieved a 96.6% accuracy.
[158]	1000 images	Images	Encoder–decoder architecture	Proposing FlameTransNet. Implementing an encoder–decoder architecture. Selecting MobileNetV2 for the encoder and DeepLabV3+ for the decoder.	Achieved an IoU, Precision, and Recall of 83.72%, 91.88%, and 90.41%, respectively.
[159]	Live data from cameras, thermopile-type sensors, and anemometers	Images, infrared, and ultrasonic	Segmentation and reconstruction	Developing an image-based diagnostic system to enhance the understanding of wildfire spread and providing tools for fire management through a 3D reconstruction of turbulent flames.	Demonstrated that the flame volume measured through image processing can reliably substitute fire thermal property measurements.
[160]	*	Images	SVM	Proposing a fire image recognition method by integrating color space information into the SIFT algorithm. Extracting fire feature descriptors using the SIFT from images, filtering noisy features using fire color space, and transforming descriptors into feature vectors. Using an Incremental Vector SVM classifier to develop the recognition model.	Achieved a 97.16% testing accuracy.
[161]	37 videos	Videos	SVM	Proposing a fire-flame detection model by defining the candidate fire regions through background subtraction and color analysis. Modeling fire behavior using spatio-temporal features and dynamic texture analysis. Classifying candidate regions using a two-class SVM classifier.	Achieved detection rates of approximately 99%.

* Information not available.

• Representative Publications:

Through an ACC graph for this category, as shown in Figure 7, only the best performer was chosen. A representative publication [160], entitled ‘The fire recognition algorithm using dynamic feature fusion and IV-SVM classifier’ and published in 2019, was chosen. This work aimed to identify flame areas using a flame recognition model based on an Incremental Vector SVM classifier. It introduces flame characteristics in color space and employs dynamic feature fusion to remove image noise from SIFT features, enhancing feature extraction accuracy. The SIFT feature extraction method incorporates flame-specific color spatial characteristics, achieving a testing accuracy of 97.16%.

Figure 7. ACC and its standard deviation (- - -) for fire and flame.

4.4. Fire and Smoke

Deep learning models excel at learning hierarchical representations of data. They can learn features at different levels of abstraction, enabling them to capture both local and global patterns associated with fire and smoke. This enhances their ability to detect fire and smoke under various environmental conditions and appearances. A total of twenty-three publications have been identified in this category, as listed in Table 4.

Table 4. List of the past works related to fire and smoke detection.

Ref	Dataset	Data Type	Method	Objective	Achievement
[162]	17,840 images	Images	CNN	Detecting forest fire smoke in real-time through using deep convolutional neural networks.	Achieved an accuracy of 95.7% on real-time forest fire smoke detection.
[163]	3000 images	Images	R-CNN	Classifying smoke columns with object detection and a DL-based approach.	Dropped the FPR to 88.7% (from 93.0%).
[164]	35,328 images	Images	Transfer learning	Improving fire and smoke recognition in still images by utilizing advanced convolutional techniques to balance accuracy and complexity.	Obtained an AUROC value of 0.949 with the test set that corresponded to a TPR and FPR of 85.3% and 3.1%, respectively.
[165]	1900 images	Images	GA-CNN	Detecting fire occurrences with high accuracy in the environment.	Achieved a 95% accuracy and 92% TPR.
[166]	3630 images	Images	CNN	Segmenting fire and smoke regions in high-resolution images based on a multi-resolution iterative quad-tree search algorithm.	Obtained a 95.9% accuracy on fire and smoke segmentation.
[167]	4326 images	Images	CNN	Building an adaptive linear feature–reuse network for rapid forest fire smoke detection.	Achieved an 87.26% mAP50 on fire and smoke detection.
[168]	15,909 images	Images	MVMNet	Detecting fire based on a value conversion attention mechanism module.	Obtained an mAP50 of 88.05% on fire detection.
[169]	14,402 images	Videos	CNN	Wildfire detection through RGB images by the CNN model.	Achieved an accuracy of 98.97% and an F1-score of 95.77% on fire and smoke detection, respectively.
[170]	7652 images	Images	R-CNN	Forest fire and smoke recognition based on an anchor box adaptive generation method.	Achieved an accuracy rate of 96.72% and an IOU of 78.96%.
[171]	1323 fire or smoke images and 3533 non-fire images	Images	R-CNN	Performing collaborative region detection and developing a grading framework for forest fire smoke using weakly supervised fine segmentation and lightweight faster-RCNN.	Achieved a 99.6% detection accuracy and 70.2% segmentation accuracy.
[172]	400,000 images	Images	BNN and RCNN	Constructing a model for early fire detection and damage area estimation for response systems.	Achieved an mAP of 27.9 for smoke and fire.
[173]	23,500 images	Images	CNN and RNN	Detecting forest fire through using a hybrid DL model.	Accomplished fire detection with 99.62% accuracy.
[174]	16,140 images	Images	CNN	Enhancing fire and smoke detection in still images through advanced convolutional methods to optimize accuracy and complexity.	Achieved 84.36% and 81.53% mean test accuracy for the fire and fire and smoke recognition tasks, respectively.
[175]	14 fire and 17 non-fire videos	Videos	R-CNN	Reducing FP detection by a smoke detection algorithm.	Attained a 99.9% accuracy in performing smoke and fire detection.
[176]	49 large images	Images	CNN	Performing active fire mapping using CNN.	Achieved a 0.84 F1-score on fire detection.
[177]	5682 images	Images	Wavelet decomposition	Detecting forest fire smoke using videos in a wavelet domain.	Achieved a 94.04% accuracy on fire detection.
[178]	1844 images	Images	MobileNetV3	Building a lightweight deep learning fire recognition algorithm that can be employed on embedded hardware.	Through experimental results, a significant reduction in the number of model parameters and inference time was achieved when compared to YOLOv4.
[179]	999 images	Satellite images	Transfer learning	Using learning without forgetting (LwF) to train the network with a new task but keeping the network’s preexisting abilities intact.	An accuracy of 91.41% was achieved by Xception with LwF on the BowFire dataset and 96.89% on the original dataset.
[180]	*	Images and videos	GS-YOLOv5	Replacing the convolutional blocks in Super-SPPF by GhostConv and using the C3Ghost module instead of the C3 module in YOLOv5 to increase speed and reduce computational complexity.	Achieved a detection accuracy of 95.9%.
[181]	3000 images	Images	YOLOv6	Enhancing model performance by integrating the Convolutional Block Attention Module (CBAM), employing the CIoU loss function, and utilizing AMP automatic mixed-precision training.	Achieved an mAP of 0.619.
[182]	450 images	Images	YOLOv5s	Integrating CA into YOLOv5, replacing YOLOv5’s SPPF module with an RFB module and enhancing the neck structure by upgrading PANet to Bi-FPN.	Improved the forest fire and smoke detection model in terms of mAP@0.5 by 5.1% compared with YOLOv5.
[183]	18,217 images	Images	YOLOv4	Proposing AERNet, a real-time fire detection network, optimizing for both accuracy and speed. Utilizing SE-GhostNet for lightweight feature extraction and an MSD module for enhanced feature emphasis. Employing decoupled heads for class and location prediction.	Achieved a 69.42% mAP50, 18.75 ms inference time, and 48 fps.
[184]	39,375 images	Images	Ensemble CNN	Using an ensemble of XceptionNet, MobileNetV2, and ResNet-50 CNN architectures for early fire prediction. Implementing fire and smoke detection using YOLO architecture known for low latency and high fps.	The smoke detection model achieved an mAP@0.5 of 0.85, while the combined model achieved an mAP@0.5 of 0.76.

* Information not available.

• Representative Publications:

Based upon an ACC graph, as shown in Figure 8, the top performer in terms of ACC in this category was the paper titled ’Forest fire and smoke detection using deep learning-based learning without forgetting’ [179]. The authors utilized transfer learning to enhance the analysis of forest smoke in satellite images. Their study introduced a dataset of 999 satellite images and employed learning without forgetting to train the network on a new task while preserving its pre-existing capabilities. In using the Xception model with LwF, the research achieved an accuracy of 91.41% on the BowFire dataset and 96.89% on the original dataset, demonstrating significant improvements in forest fire and smoke detection accuracy.

Figure 8. ACC and its standard deviation (- - -) for fire and smoke.

Based on the plot, Ref. [168] was the second-best performer with the second-highest score of almost thirty-five. This publication, entitled ‘Fast forest fire smoke detection using MVMNet’, was published in 2022. The paper proposed multi-oriented detection based on a value conversion-attention mechanism module and mixed-NMS for smoke detection. They obtained the forest fire multi-oriented detection dataset, which includes 15,909 images. The mAP and mAP50 achieved were 78.92% and 88.05%, respectively.

4.5. Applications of Robots in Fire Detection and Extinguishing

Robots equipped with cameras or vision sensors can capture images or video footage of their surroundings. Deep learning models trained on fire datasets can analyze this visual input, enabling the robot to detect the presence of fire. CNNs are commonly used for image-based fire detection in robot systems.

Deep learning models can be employed to enhance the robot’s decision-making capabilities during fire extinguishing operations. By training the model on datasets that include fire dynamics, robot behavior, and firefighting strategies, the robot can learn to make informed decisions on approaches such as selecting the appropriate firefighting equipment, assessing the fire’s intensity, or planning extinguishing maneuvers. There exist very few examples where robots are utilized in actual fields for forest fire detection. To highlight the potential of robots in fire detection and extinguishing, indoor and outdoor scenarios, in addition to wildfires, are also included. Past research efforts related to fire detection and extinguishing with the help of robots are listed in Table 5.

Table 5. List of the past works related to the utilization of robots in fire detection and extinguishing.

Ref	Environment	Robot Type	Objectives	Achievements
[185]	Outdoor	UGV	To build a four-drive articulated tracked fire extinguishing robot that can flexibly perform fire detection and fire extinguishing.	Designed a firefighting robot that can be operated remotely to control its movements and can spray through its cannon.
[186]	Indoor/outdoor	UGV	Building a firefighter intervention architecture that consists of several sensing devices, a navigation platform (an autonomous ground wheeled robot), and a communication/localization network.	Achieved an accuracy of 73% and precision of 99% in detecting fire points.
[187]	Indoor/outdoor	UGV	Building a smart sensor network-based autonomous fire extinguish robot using IoT.	Successfully demonstrated the robot working on nine different occasions.
[188]	Indoor/outdoor	UGV	Developing a small wheel-foot hybrid firefighting robot for infrared visual fire recognition.	Achieved an average recognition rate of 97.8% with the help of a flame recognition algorithm.
[189]	Buildings	UGV	Building an autonomous firefighter robot with a localized fire extinguisher.	The robot, which is equipped with six flame sensors, can detect flame instantly and can extinguish fire with the help of sand.
[190]	Outdoor	UGV	Building an autonomous system for wildfire and forest fire early detection and control.	The autonomous firefighting robot equipped with a far infrared sensor and turret can detect and extinguish small fires within range.
[191]	Indoor/outdoor	UGV	Performing fire extinguishing without the need for firefighters.	Extinguished fire at a maximum distance of 40 cm from the fire.
[192]	Forest	UAV	Building wildfire detection solution based on unmanned aerial vehicle-assisted Internet of Things (UAV-IoT) networks.	The rate of detecting a 2.5 km2 fire was more than 90%.
[193]	Forest	UAV	Detecting forest fires through the use of a new color index.	A detection precision of 96.82% is achieved.
[194]	Outdoor	UAV	Exploring the potential of DL models, such as YOLO and R-CNN, for forest fire detection using drones.	An mAP@0.5% of 90.57% and 89.45% were achieved by Faster R-CNN and YOLOv8n, respectively.
[195]	Outdoor	UAV	Proposing a low-cost UAV with extended MobileNet deep learning for classifying forest fires. Share fire detection and GPS location with state forest departments for a timely response.	Achieved an accuracy of 97.26%.
[196]	Outdoor	UAV	Proposing a novel wildfire identification framework that adaptively learns modality-specific and shared features. Utilizing parallel encoders to extract multiscale RGB and TIR features, integrating them into a fusion feature layer.	The proposed method achieved an average improvement of 6.41% and 3.39% in IoU and F1-score, respectively, compared to the second-best RGB-T semantic segmentation method.
[197]	Outdoor	UAV	Proposing a two-stage framework for fire detection and geo-localization. Compiling a large dataset from several sources to capture the various visual contexts related to fire scenes. Investigating YOLO models.	Achieved an mAP50 of 0.71 and an F1-score of 0.68.
[198]	Outdoor	UAV	Introducing the UAV platform “WILD HOPPER,” a 600-liter capacity system designed specifically for forest firefighting.	Achieved a payload capacity that addresses the common limitations of electrically powered drones, which are typically restricted to fire monitoring due to insufficient lifting power.
[199]	Outdoor	UAV	To explore the integration of fire extinguishing balls with drone and remote-sensing technologies as a complementary system to traditional firefighting methods.	Controlled experiments were conducted to assess the effectiveness and efficiency of fire extinguishing balls.
[200]	Outdoor	UAV	To promote the use of UAVs in firefighting by introducing a metal alloy rotary-wing UAV equipped with a payload drop mechanism for delivering fire-extinguishing balls to inaccessible areas.	Examined the potential of UAVs equipped with a payload drop mechanism for fire-fighting operations.
[201]	Outdoor	UAV	To propose a concept of deploying drone swarms in fire prevention, surveillance, and extinguishing tasks.	Developed a concept for utilizing drone swarms in firefighting, addressing issues reported by firefighters and enhancing both operational efficiency and safety.
[202]	Outdoor	UAV	To improve the Near-Field Computer Vision system for an intelligent fire robot to accurately predict the falling position of jet trajectories during fire extinguishing.	The system for intelligent fire extinguishing achieved a reduction in the average prediction error from 1.36 m to 0.1 m and a reduction in error variance from 1.58 m to 0.13 m in terms of predicting jet-trajectory falling positions.

• Representative Publications:

The ACC for papers in this category is illustrated in Figure 9. Two papers were chosen as representative publications from this category. One of the selected papers is entitled ‘The Role of UAV-IoT Networks in Future Wildfire Detection’. In this paper, a novel wildfire detection solution based on unmanned aerial vehicle-assisted Internet of Things (UAV-IoT) networks was proposed [192]. The main objectives were to study the performance and reliability of the UAV-IoT networks for wildfire detection and to present a guideline to optimize the UAV-IoT network to improve fire detection probability under limited system cost budgets. Discrete-time Markov chain analysis was utilized to compute the fire detection and false-alarm probabilities. Numerical results suggested that, given enough system cost, UAV-IoT-based fire detection can offer a faster and more reliable wildfire detection solution than state-of-the-art satellite imaging techniques.

Figure 9. ACC and its standard deviation (- - -) for applications of robots in fire detection and extinguishing.

The second paper that was chosen is titled ’A Survey on Robotic Technologies for Forest Firefighting: Applying Drone Swarms to Improve Firefighters’ Efficiency and Safety’ [201]. In this paper, a concept for deploying drone swarms in fire prevention, surveillance, and extinguishing tasks was proposed. The objectives included evaluating the effectiveness of drone swarms in enhancing operational efficiency and safety in firefighting missions, as well as in addressing the challenges reported by firefighters. The system utilizes a fleet of homogeneous quad-copters equipped for tasks such as surveillance, mapping, and monitoring. Moreover, the paper discussed the potential of this drone swarm system to improve firefighting operations and outlined challenges related to scalability, operator training, and drone autonomy.

5. Discussion

Fire, smoke, and flame detection and their extinguishing are considered challenging problems due to the complex behavior and dynamics of fire, which makes them difficult to predict and control. Based on the literature, we identified the following important factors.

5.1. Variability in Fire, Smoke, and Flame Types and Appearances

In our analysis, almost all articles were found to have utilized modern resources and technologies to make the proposed approaches as effective as possible. We found several articles in the literature that focused on variation based on the type, color, size, and intensity (Table 6).

Table 6. Methods of handling variations in fire, flame, and smoke.

Nature	Methods
Fire	Infrared [57,188], convex hulls [86], deep learning [67,76,83,94,175], color probabilities and motion features [84], multi-task learning [66], ensemble learning [73], semantic [85], optimization [165], Markov chain [192], support vector machine [53,59], visible infrared imaging [60], visible-NIR [159]
Flame	Deep learning [49,94], support vector machine [160], spatio-temporal features and SVM [161], infrared [190], visible-NIR [159], spatio-temporal features and deep learning [175]
Smoke	Deep learning [147,148,172], stereo camera [124], transformer [128]

Our analysis found that forest fire detection and extinguishing systems underscore the significant advancements made in this field, particularly in leveraging modern resources and technologies such as deep neural networks (DNNs). These technologies have proven essential in addressing the variability in fire, smoke, and flame types; appearances; and intensities, enabling more accurate detection and response.

5.2. Response Time

The ability to detect fires early is crucial for prompt intervention and minimizing potential damage. Many studies have emphasized early detection, but there is often a lack of concrete evidence regarding the computational efficiency and real-world effectiveness of these methods, particularly in forest fire scenarios. A common issue is the lack of practical testing and transparency. For instance, [62] tested a GMM to detect the smoke signatures on plumes, achieving a detection rate of 18–20 fps, but they did not test it in real forest fire scenarios, limiting practical evidence. Similarly, [78] conducted tests with a controlled small fire but did not provide time metrics for real-time applicability. The authors in [164] utilized a dataset collected over 274 days from nine real surveillance cameras mentioning “early detection” without specific metrics, making it difficult to assess it for practical effectiveness. In [78], the authors claimed to detect 78% of wildfires earlier than the VIIRS active fire product, but they did not include explicit time measurements, hindering a thorough evaluation of its early-detection capabilities.

Some studies provided more concrete data on the speed and efficiency of their detection methods. For example, [73] used aerial image analysis with ensemble learning to achieve an inference time of 0.018 s, showcasing rapid detection potential. The multi-oriented detection method in [168] achieved a frame rate of 122 fps, which was higher than YOLOv5 (156 fps), though with a lower mean average precision (mAP). Another study used a dataset of 1135 images, reporting an inference time of 2 s for forest fire segmentation using vision transformers [70]. The deep neural network-based approach (AddNet) saved 12.4% time compared to a regular CNN, and it was tested on a dataset of 4000 images [81]. The performance of EfficientDet, YOLOv3, SSD, and Faster R-CNN was evaluated on a dataset of 17,840 images, with YOLOv3 being the fastest at 27 fps [162]. The method in [174], evaluated with a dataset of 16,140 images, achieved a processing time per image of 0.42 s, which was claimed to be four times faster than the compared models.

Although “early detection” is a frequently used term, specific, quantifiable metrics to support these claims are often lacking. The reviewed studies highlight various methods and technologies, but the need for comprehensive, real-world testing and transparent reporting remains.

5.3. Environmental Contextual and Adaptability

The effectiveness of fire detection systems under various environmental conditions is critical for their accuracy and reliability. Environmental factors such as weather, terrain, and other influences can significantly impact performance, leading to false positives or missed detection.

Environmental factors like cloud cover and weather conditions pose significant challenges for fire detection systems. For example, [75] achieved a 92% detection rate in clear weather but only 56% in cloudy conditions using multi-sensor satellite imagery from Sentinel-1 and Sentinel-2. Similarly, [78] utilized geostationary weather satellite data and proposed max aggregation to reduce cloud and smoke interference, enhancing detection accuracy. Not all studies addressed varying weather conditions comprehensively. Ref. [150] used an unsupervised method without specific solutions for different forecast conditions, demonstrating a lack of robustness in dynamic environments. Additionally, [115] highlighted that wildfire detection probability by MODIS is significantly influenced by factors such as daily relative humidity, wind speed, and altitude, underscoring the need for adaptable detection systems.

False positives are a critical issue in fire detection systems as they can lead to unnecessary alarms and resource deployment. Various strategies have been employed to mitigate this issue. For instance, [72] proposed dividing detected regions into blocks and using multidimensional texture features with a clustering approach to filter out false positives accurately. This method focuses on improving the specificity of the detection system. Other approaches include threshold optimization, as seen in [57], where fires with more than a 30% confidence level were selected to reduce false alarms in the MODIS14 dataset. Ref. [62] attempted to discriminate between smoke, fog, and clouds by converting the RGB color space to hue, saturation, and luminance; though the study lacked a thorough evaluation and comparison of results.

Combining traditional and deep learning methods has shown promise in improving detection accuracy. Ref. [121] integrated a hand-designed smoke detection model with a deep learning model, successfully reducing the false negative and false positive rates, thereby enhancing smoke recognition accuracy. The authors in [147] addressed the challenge of non-smoke images containing features similar to smoke, such as colors, shapes, and textures, by proposing multiscale convolutional layers for scale-invariant smoke recognition.

Detection in fog or dust conditions presents additional challenges. The authors in [151] compared their approach with other methods, including SVM, Bayes classifier, fuzzy c-means, and Back Propagation Neural Network, and they demonstrated the lowest false alarm rate in wildfire smoke detection under heavy fog. Further advancements include the use of quasi-dynamic features and dual tree-complex wavelet transform with elastic net processing, as proposed by [177], to handle disturbances like fog and haze. Similarly, [148] developed a deep convolutional neural network to address variations in image datasets, such as clouds, fog, and sandstorms, achieving an average accuracy of 97%. However, they noted a performance degradation when testing on wildfire smoke compared to nearby smoke, indicating the need for more specialized training datasets.

5.4. Extinguishing Efficiency

Most of the development of firefighting robots has mainly focused on indoor and smooth outdoor environments, limiting their use in rugged terrains like forests. These robots are designed to assist in firefighting, but their effectiveness in actual forest environments is largely untested. Most existing firefighting UGVs are suited for smooth surfaces and controlled conditions, such as urban areas, and are equipped with fire suppression systems and sensors. However, they are not optimized for the unpredictable conditions of forests.

Some pioneering efforts are being made to develop technologies specifically for forest environments. For instance, a UAV platform with a 600-L payload capacity and equipped with thermographic cameras and navigation systems has been proposed, but it has not been fully tested in real-world conditions [198]. Another study explored the use of fire extinguishing balls deployed from unmanned aircraft systems; though practical effectiveness remained uncertain due to limited integration evidence [199,200]. Research has also focused on robotized surveillance with conventional, multi-spectral, and thermal cameras, primarily for situational awareness and early detection [201]. However, there is a gap in integrating autonomous systems for direct fire suppression, with most efforts centered on surveillance rather than active firefighting.

While there are promising developments, forest firefighting robots are still in the early stages of research and development. Most current technologies are designed for controlled environments and have not been extensively tested in forest conditions. Therefore, their efficiency and practical effectiveness cannot be validated due to a lack of evidence and comprehensive testing.

5.5. Compliance and Standards

The use of UAVs for forest fire detection and extinguishing offers advantages like rapid deployment, real-time data acquisition, and access to hard-to-reach areas. However, integrating UAVs into these applications presents challenges, particularly regarding compliance with regional regulations and safety standards. For instance, in Canada, UAV operators must obtain a pilot license, maintain a line of sight with the UAV, and avoid flying near forest fires [130]. These regulations, while essential for safety, can limit the effectiveness and operational scope of UAV-based systems. Our review found a lack of focus on developing UAV hardware that complies with these regulatory frameworks, highlighting the need for compliant technologies that can operate safely and legally across different regions.

6. Recommendations for Future Research

A review of the current literature on forest fire detection and extinguishing systems revealed several key areas where further research and development are needed. Addressing these gaps will not only enhance the effectiveness of these systems but also ensure their safe and compliant integration into existing fire management frameworks. Below are three primary gaps that were identified, along with corresponding recommendations for future research.

6.1. Recommendation 1: Integration of Real-Time Data Processing and Decision-Making Algorithms

Gap: Current research often focuses on the capabilities of UAV systems for data collection but there is a lack of emphasis on the integration of real-time data processing and decision-making algorithms [82,130]. This integration is crucial for enabling UAVs to respond promptly and accurately to detecting fires, especially in rapidly changing environments.

Recommendation: Future research should concentrate on developing and integrating advanced algorithms capable of real-time data processing [174] and decision making [202]. This includes machine learning and AI techniques that can analyze sensor data on-the-fly, identify potential fire hazards, and make autonomous decisions regarding navigation and intervention. Researchers should explore how these algorithms can be implemented efficiently on UAV platforms, considering constraints like computational power and energy consumption [110,169].

6.2. Recommendation 2: Effectiveness and Autonomy in Real-World Conditions

Gap: Although numerous UAV systems have been proposed for forest fire detection and extinguishing, many have not been extensively tested or validated in real-world conditions [65,73,198]. This lack of field testing raises concerns about the practical effectiveness, functionality, and autonomy of these systems in the diverse and challenging environments typical of forest fires.

Recommendation: There is a need for comprehensive field trials and simulations that replicate the conditions of actual forest fires. Future research should focus on developing and testing UAV systems in varied and dynamic environments to assess their performance in detecting and responding to fires. This includes testing the systems’ navigation capabilities, sensor accuracy, and overall operational reliability.

6.3. Recommendation 3: Human–Robot Interactions and Collaboration

Gap: While UAVs offer advanced surveillance and early detection capabilities, there is limited research on how these systems can effectively interact and collaborate with human firefighters. Our analysis found no article that discusses the HRI for the forest fire. Ensuring seamless HRI is crucial for optimizing the use of UAVs in firefighting, including coordinating actions with ground teams and ensuring the safety and efficiency of operations.

Recommendation: Future research should explore the development of systems and protocols that facilitate effective HRI in the context of forest firefighting. This includes designing intuitive interfaces and communication systems that allow human operators to easily control and monitor UAVs. Additionally, research should focus on developing collaborative frameworks where UAVs and human firefighters can work together, leveraging each other’s strengths. For example, UAVs can provide real-time aerial data to ground teams, enhancing situational awareness and guiding decision-making processes [58]. Studies should also address the psychological and ergonomic aspects of HRI, ensuring that the introduction of UAVs does not overwhelm or distract human operators but rather complements their efforts.

7. Conclusions

Automatic fire detection in forests is a critical aspect of modern wildfire management and prevention. In this paper, through the PRISMA framework, we surveyed a total of 155 journal papers that concentrated on fire detection using image processing, computer vision, deep learning, and machine learning for the time span of 2013–2023. The literature review was mainly classified into four categories: fire, smoke, fire and flame, and fire and smoke. We also categorized the literature based on their applications in real fields for fire detection, fire extinguishing, or a combination of both. We observed an exponential increase in the number of publications from 2018 onward; however, very limited research has been conducted in the utilization of robots for the detection and extinguishing of fire in hazardous environments. We predict that, with the increasing number of fire incidents in the forests and with the increased popularity of robots, the trend of autonomous systems for fire detection and extinguishing will thrive. We hope that this research work can be used as a guidebook for researchers who are looking for recent developments in forest fire detection using deep learning and image processing to perform further research in this domain.