An Explainable AI-Based Modified YOLOv8 Model for Efficient Fire Detection
Abstract
:1. Introduction
- In this work, we modified the YOLOv8 architecture for the development of an efficient automated fire and smoke detection system, achieving impressive accuracy in fire and smoke detection. Moreover, a comprehensive fire dataset was created, consisting of 4301 labeled images. Each image was carefully annotated, providing a valuable resource for improving the performance of models in identifying fire and smoke in various conditions.
- We also utilized EigenCAM to explain and visualize the results of the proposed model, highlighting the image areas that most influenced the model’s decisions. This approach enhances the understanding of the model’s behavior, thereby improving the interpretability and transparency of its predictions.
2. Related Works
3. Methodology
Algorithm 1 Fire Detection Model Workflow |
|
3.1. Dataset Description and Preprocessing
3.2. Model Architecture and Modifications
3.3. Changes to the YOLOv8 Backbone
3.4. The Training and Success Measure Process
3.5. Improvements after Using C2f Layer
4. Result Analysis
4.1. Evaluation Metrics
4.2. Results of the Proposed Model
4.3. Comparing at YOLOv7 and YOLOv8
4.4. Impact of Hyperparameter Tuning on YOLOv8
4.5. Activation Function
4.6. Performance in a Range of Fire Situations
5. Explainability with EigenCAM
5.1. Eigen Class Activation Maps (EigenCAM)
5.2. Implementation and Results of EigenCAM
6. Discussion
7. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
Adam | Adaptive Moment Estimation |
CCTV | Closed-Circuit Television |
C2F | Context-To-Flow |
CNN | Convolutional Neural Networks |
CVAT | Computer Vision Annotation Tool |
DBSCAN | Density-Based Spatial Clustering Of Applications With Noise |
EigenCAM | Eigen Class Activation Maps |
GFLOPS | Giga Floating-Point Operations Per Second |
IR | Infrared |
LeakyReLU | Leaky Rectified Linear Unit |
LSTM | Long Short-Term Memory |
mAP | Mean Average Precision |
MQTT | Message Queuing Telemetry Transport |
R-CNN | Region-Based Convolutional Neural Network |
ReLU | Rectified Linear Unit |
RNN | Recurrent Neural Network |
SECSP | Spatially Enhanced Contextual Semantic Parsing |
SGD | Stochastic Gradient Descent |
SIoU | Scylla Intersection over Union |
SLIC | Simple Linear Iterative Clustering |
Softmax | Softargmax Or Normalized Exponential Function |
SPPF | Spatial Pyramid Pooling Fast |
SRoFs | Suspected Regions Of Fire |
SSD | Single-Shot Detector |
SVD | Singular Value Decomposition |
Tanh | Hyperbolic Tangent |
VGG | Visual Geometry Group |
YOLO | You Only Look Once |
BEMRF | Boundary Enhancement and MultiScale Refinement Fusion |
GLCA | Global–Local Context Aware |
References
- Saponara, S.; Elhanashi, A.; Gagliardi, A. Real-time video fire/smoke detection based on CNN in antifire surveillance systems. J. Real-Time Image Process. 2021, 18, 889–900. [Google Scholar] [CrossRef]
- Abdusalomov, A.; Baratov, N.; Kutlimuratov, A.; Whangbo, T. An improvement of the fire detection and classification method using YOLOv3 for surveillance systems. Sensors 2021, 21, 6519. [Google Scholar] [CrossRef] [PubMed]
- Barmpoutis, P.; Stathaki, T.; Dimitropoulos, K.; Grammalidis, N. Early fire detection based on aerial 360-degree sensors, deep convolution neural networks and exploitation of fire dynamic textures. Remote Sens. 2020, 12, 3177. [Google Scholar] [CrossRef]
- Avazov, K.; Mukhiddinov, M.; Makhmudov, F.; Cho, Y. Fire detection method in smart city environments using a deep-learning-based approach. Electronics 2022, 11, 73. [Google Scholar] [CrossRef]
- Xu, W.; Feng, Z.; Wan, Q.; Xie, Y.; Feng, D.; Zhu, J.; Liu, Y. Building Height Extraction From High-Resolution Single-View Remote Sensing Images Using Shadow and Side Information. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2024, 17, 6514–6528. [Google Scholar] [CrossRef]
- Chen, H.; Feng, D.; Cao, S.; Xu, W.; Xie, Y.; Zhu, J.; Zhang, H. Slice-to-slice context transfer and uncertain region calibration network for shadow detection in remote sensing imagery. ISPRS J. Photogramm. Remote Sens. 2023, 203, 166–182. [Google Scholar] [CrossRef]
- Zhu, J.; Zhang, J.; Chen, H.; Xie, Y.; Gu, H.; Lian, H. A cross-view intelligent person search method based on multi-feature constraints. Int. J. Digit. Earth 2024, 17, 2346259. [Google Scholar] [CrossRef]
- Cao, S.; Feng, D.; Liu, S.; Xu, W.; Chen, H.; Xie, Y.; Zhang, H.; Pirasteh, S.; Zhu, J. BEMRF-Net: Boundary Enhancement and MultiScale Refinement Fusion for Building Extraction from Remote Sensing Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2024, 17, 16342–16358. [Google Scholar] [CrossRef]
- Xie, Y.; Zhan, N.; Zhu, J.; Xu, B.; Chen, H.; Mao, W.; Luo, X.; Hu, Y. Landslide extraction from aerial imagery considering context association characteristics. Int. J. Appl. Earth Obs. Geoinf. 2024, 131, 103950. [Google Scholar] [CrossRef]
- Ibraheem, M.K.I.; Mohamed, M.B.; Fakhfakh, A. Forest Defender Fusion System for Early Detection of Forest Fires. Computers 2024, 13, 36. [Google Scholar] [CrossRef]
- Liu, J.; Yin, J.; Yang, Z. Fire Detection and Flame-Centre Localisation Algorithm Based on Combination of Attention-Enhanced Ghost Mode and Mixed Convolution. Appl. Sci. 2024, 14, 989. [Google Scholar] [CrossRef]
- Wang, B.; Zhao, X.; Zhang, Y.; Song, Z.; Wang, Z. A capacitive particle-analyzing smoke detector for very early fire detection. Sensors 2024, 24, 1692. [Google Scholar] [CrossRef] [PubMed]
- Pincott, J.; Tien, P.; Wei, S.; Calautit, J. Indoor fire detection utilizing computer vision-based strategies. J. Build. Eng. 2023, 61, 105154. [Google Scholar] [CrossRef]
- Muhammad, K.; Ahmad, J.; Mehmood, I.; Rho, S.; Baik, S.W. Convolutional neural networks based fire detection in surveillance videos. IEEE Access 2018, 6, 18174–18183. [Google Scholar] [CrossRef]
- de Venancio, P.; Lisboa, A.; Barbosa, A. An automatic fire detection system based on deep convolutional neural networks for low-power, resource-constrained devices. Neural Comput. Appl. 2022, 34, 15349–15368. [Google Scholar] [CrossRef]
- Park, J.H.; Lee, S.; Yun, S.; Kim, H.; Kim, W.T. Dependable fire detection system with multifunctional artificial intelligence framework. Sensors 2019, 19, 2025. [Google Scholar] [CrossRef]
- Park, M.; Ko, B.C. Two-step real-time night-time fire detection in an urban environment using Static ELASTIC-YOLOv3 and Temporal Fire-Tube. Sensors 2020, 20, 2202. [Google Scholar] [CrossRef] [PubMed]
- Muhammad, M.B.; Yeasin, M. Eigen-CAM: Visual Explanations for Deep Convolutional Neural Networks. SN Comput. Sci. 2021, 2, 47. [Google Scholar] [CrossRef]
- Ahn, Y.; Choi, H.; Kim, B. Development of early fire detection model for buildings using computer vision-based CCTV. J. Build. Eng. 2023, 65, 105647. [Google Scholar] [CrossRef]
- Yunusov, N.; Islam, B.M.S.; Abdusalomov, A.; Kim, W. Robust Forest Fire Detection Method for Surveillance Systems Based on You Only Look Once Version 8 and Transfer Learning Approaches. Processes 2024, 12, 1039. [Google Scholar] [CrossRef]
- Li, P.; Zhao, W. Image fire detection algorithms based on convolutional neural networks. Case Stud. Therm. Eng. 2019, 19, 100625. [Google Scholar] [CrossRef]
- Muhammad, K.; Ahmad, J.; Baik, S. Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 2019, 288, 30–42. [Google Scholar] [CrossRef]
- Kim, B.; Lee, J. A video-based fire detection using deep learning models. Appl. Sci. 2019, 9, 2862. [Google Scholar] [CrossRef]
- Xie, Y.; Zhu, J.; Cao, Y.; Zhang, Y.; Feng, D.; Zhang, Y.; Chen, M. Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features. IEEE Access 2020, 8, 81904–81917. [Google Scholar] [CrossRef]
- Valikhujaev, Y.; Abdusalomov, A.; Cho, Y. Automatic fire and smoke detection method for surveillance systems based on dilated CNNs. Atmosphere 2020, 11, 1241. [Google Scholar] [CrossRef]
- Sheng, D.; Deng, J.; Xiang, J. Automatic smoke detection based on SLIC-DBSCAN enhanced convolutional neural networks. Sensors 2020, 20, 5608. [Google Scholar] [CrossRef]
- Lin, G.; Zhang, Y.; Xu, G.; Zhang, Q. Smoke detection on video sequences using 3D convolutional neural networks. Fire Technol. 2019, 55, 1827–1847. [Google Scholar] [CrossRef]
- Pundir, A.; Raman, B. Dual deep learning model for image based smoke detection. Fire Technol. 2019, 55, 2419–2442. [Google Scholar] [CrossRef]
- Zhao, Y.; Zhang, H.; Zhang, X.; Chen, X. Fire smoke detection based on target-awareness and depthwise convolutions. Multimed. Tools Appl. 2020, 80, 27407–27421. [Google Scholar] [CrossRef]
- Dilshad, N.; Khan, T.; Song, J. Efficient deep learning framework for fire detection in complex surveillance environment. Comput. Syst. Sci. Eng. 2023, 46, 749–764. [Google Scholar] [CrossRef]
- Jacob, S.; Francesco. What is YOLOv8? A Complete Guide. Roboflow Blog. Available online: https://blog.roboflow.com/what-is-yolov8 (accessed on 1 September 2024).
- Fahy, R.; Petrillo, J. Firefighter Fatalities in the US in 2022. 2022. Available online: https://www.usfa.fema.gov/downloads/pdf/publications/firefighter-fatalities-2022.pdf (accessed on 1 June 2024).
- Solawetz, J.; Francesco. What is YOLOv8? The Ultimate Guide. 2023. Available online: https://blog.roboflow.com/whats-new-in-yolov8/ (accessed on 19 July 2024).
- Mehedi, S.T.; Anwar, A.; Rahman, Z.; Ahmed, K. Deep transfer learning based intrusion detection system for electric vehicular networks. Sensors 2021, 21, 4736. [Google Scholar] [CrossRef] [PubMed]
- Inside, A.I. Introduction to Deep Learning with Computer Vision—Learning Rates & Mathematics—Part 1. 2020. Available online: https://medium.com/hitchhikers-guide-to-deep-learning/13-introduction-to-deep-learning-with-computer-vision-learning-rates-mathematics-part-1-4973aacea801 (accessed on 3 September 2024).
Ref. | Algorithm | Accuracy | Limitations | Future Directions |
---|---|---|---|---|
[19] | YOLO | 0.97 (Recall), 0.91 (Precision) | Limited to CCTV range | Incorporate temperature and sound |
[13] | R-CNN, SSD | 95% (Fire), 62% (Smoke) | Not tested outdoors | Test in outdoor environments |
[4] | YOLOv4 | 98.8% | Not suitable for large areas | Apply to larger areas |
[2] | YOLOv3 | 98.9% | Errors with electrical lamps | Improve detection at night |
[22] | CNN | 98.5% | Balance between accuracy and false alarms | Optimize accuracy vs. false alarms |
[14] | CNN | 94.43% | High false alarms | Improve tuning for real-world scenarios |
[24] | CNN | 97.94% (Best) | Issues with multiple moving objects | Integrate with IoT for real-time detection |
[21] | CNN | 83.7% | Lower accuracy, delayed alarms | Improve detection accuracy and speed |
[1] | YOLOv2 | 96.82% | False positives in challenging environments | Connect with cloud facilities |
[25] | CNN | 99.53% | Early detection issues in clouds | Develop lightweight, robust model |
[26] | SLIC-DBSCAN | 87.85% | High false positives | Improve sensitivity, reduce FPR |
[27] | R-CNN, 3D CNN | 95.23% | Small training and validation dataset | Collect more diverse dataset |
[28] | CNN | 97.49% | Affected by clouds, fog | Extend to real-time fire detection |
[29] | Depthwise | 93.98% | Reduced accuracy in varying conditions | Balance speed and accuracy |
[15] | CNN | mAP 73.98%, F1 0.724 | Generalization issues, false alarms | Explore layer pruning |
[23] | R-CNN, LSTM | mAP 88.3%, Smoke 87.5% | False detection with non-fire objects | Enhance dataset, improve accuracy |
[30] | E-FireNet | Acc 0.98, F1 0.99 | Limited, monotonous datasets | Expand dataset, improve generalization |
[20] | YOLOv8 and TranSDet | Acc 97%, F1 96.3% | Occasionally misidentifies the sun and electric lights as fire | Expand dataset to address the limitations |
Parameters | Values |
---|---|
Epochs | 350 |
Batch Size | 9 |
Image Size | 736 |
Optimizer | Adamax |
Learning Rate | 0.001 |
Momentum | 0.995 |
Weight Decay | 0.00005 |
Validation | True |
Rect | False |
Warmup Epochs | 4.0 |
Single Class | False |
Patience | 0 |
Model | Precision | Recall | mAP@50 | F1-Score |
---|---|---|---|---|
YOLOv7 | 75.1% | 98.0% | 76.4% | 77.0% |
YOLOv8 (Default) | 95.0% | 92.5% | 96.5% | 94.0% |
Proposed Model (YOLOv8) | 97.9% | 97.2% | 99.1% | 98.0% |
No. | Model Type | Activation Function | Epochs | Optimizer | Layers | Momentum | F1-Score |
---|---|---|---|---|---|---|---|
1 | YOLOv8l | LeakyReLU | 350 | Adamax | 365 | 0.995 | 84% |
2 | YOLOv8l | Softmax | 250 | Adamax | 365 | 0.994 | 12% |
3 | YOLOv8l | Tanh | 200 | AdamW | 365 | 0.993 | 89% |
4 | YOLOv8n | Tanh | 200 | AdamW | 225 | 0.993 | 87% |
5 | YOLOv8l | ReLU | 200 | AdamW | 365 | 0.999 | 88% |
6 | YOLOv8n | ReLU | 200 | Adamax | 225 | 0.999 | 84% |
7 | YOLOv8l | Mish | 200 | SGD | 365 | 0.995 | 89% |
8 | YOLOv8n | Mish | 200 | AdamW | 225 | 0.995 | 88% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hasan, M.W.; Shanto, S.; Nayeema, J.; Rahman, R.; Helaly, T.; Rahman, Z.; Mehedi, S.T. An Explainable AI-Based Modified YOLOv8 Model for Efficient Fire Detection. Mathematics 2024, 12, 3042. https://doi.org/10.3390/math12193042
Hasan MW, Shanto S, Nayeema J, Rahman R, Helaly T, Rahman Z, Mehedi ST. An Explainable AI-Based Modified YOLOv8 Model for Efficient Fire Detection. Mathematics. 2024; 12(19):3042. https://doi.org/10.3390/math12193042
Chicago/Turabian StyleHasan, Md. Waliul, Shahria Shanto, Jannatun Nayeema, Rashik Rahman, Tanjina Helaly, Ziaur Rahman, and Sk. Tanzir Mehedi. 2024. "An Explainable AI-Based Modified YOLOv8 Model for Efficient Fire Detection" Mathematics 12, no. 19: 3042. https://doi.org/10.3390/math12193042
APA StyleHasan, M. W., Shanto, S., Nayeema, J., Rahman, R., Helaly, T., Rahman, Z., & Mehedi, S. T. (2024). An Explainable AI-Based Modified YOLOv8 Model for Efficient Fire Detection. Mathematics, 12(19), 3042. https://doi.org/10.3390/math12193042