Multi-Scale Semantic Segmentation for Fire Smoke Image Based on Global Information and U-Net
Abstract
:1. Introduction
2. Related Work
2.1. Neural Attention
2.2. Transformer
3. Materials and Methods
3.1. Smoke–U-Net
3.2. Multi-Scale Residual Group Attention (MRGA)
3.3. Global Features Module (GFM)
3.4. Loss Function
4. Results
4.1. Experimental Platform
4.2. Dataset
4.3. Evaluation Index
4.4. Results of Train and Validation
5. Discussion
5.1. Ablation Experiment
5.2. Comparative Experiment of Multi-Scale Segmentation
5.3. Comprehensive Experiments
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional network for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 3431–3440. [Google Scholar]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef] [PubMed]
- Zhao, H.; Shi, J.; Qi, X.; Wang, X.; Jia, J. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; IEEE Computer Society Press: Los Alamitos, CA, USA, 2017; pp. 2881–2890. [Google Scholar]
- Fu, J.; Liu, J.; Tian, H.; Li, Y.; Bao, Y.; Fang, Z.; Lu, H. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 5–20 June 2019; IEEE Computer Society Press: Los Alamitos, CA, USA, 2019; pp. 3141–3149. [Google Scholar]
- Zhou, T.; Li, L.; Li, X.; Feng, C.M.; Li, J.; Shao, L. Group-wise learning for weakly supervised semantic segmentation. IEEE Trans. Image Processing 2021, 31, 799–811. [Google Scholar] [CrossRef] [PubMed]
- Wang, W.; Zhou, T.; Yu, F.; Dai, J.; Konukoglu, E.; Van Gool, L. Exploring cross-image pixel contrast for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021. [Google Scholar]
- Zhou, T.; Wang, W.; Konukoglu, E.; Van Gool, L. Rethinking Semantic Segmentation: A Prototype View. arXiv 2022, arXiv:2203.15102v2. [Google Scholar]
- Appana, D.K.; Islam, R.; Khan, S.A.; Kim, J.M. A video-based smoke detection using smoke flow pattern and spatial-temporal energy analyses for alarm systems. Inf. Sci. 2017, 418, 91–101. [Google Scholar] [CrossRef]
- Zhao, Z.M.; Zhu, Z.L.; Liu, M. Fuzzy c-means clustering method for image segmentation insensitive to class size. Laser Optoelectron. Prog. 2020, 57, 56–65. [Google Scholar]
- Wang, R.; Yao, A.; Yang, R. Application of multi-scale image edge detection based on wavelet transform modulus maxima in smoke image. Foreign Electron. Meas. Technol. 2020, 39, 63–67. [Google Scholar]
- Peng, Y.; Wang, Y. Real-time forest smoke detection using hand-designed features and deep learning. Comput. Electron. Agric. 2019, 167, 105029. [Google Scholar] [CrossRef]
- Wu, Y.; Chen, M.; Wo, Y.; Han, G. Video smoke detection base on dense optical flow and convolutional neural network. Multimed. Tools Appl. 2021, 80, 35887–35901. [Google Scholar] [CrossRef]
- Khan, S.; Muhammad, K.; Hussain, T.; Del Ser, J.; Cuzzolin, F.; Bhattacharyya, S.; Akhtar, Z.; de Albuquerque, V.H. DeepSmoke: Deep learning model for smoke detection and segmentation in outdoor environments. Expert Syst. Appl. 2021, 182, 115125. [Google Scholar] [CrossRef]
- Yuan, F.; Zhang, L.; Xia, X.; Huang, Q.; Li, X. A gated recurrent network with dual classification assistance for smoke semantic segmentation. IEEE Trans. Image Processing 2021, 30, 4409–4422. [Google Scholar] [CrossRef] [PubMed]
- Gupta, T.; Liu, H.; Bhanu, B. Early Wildfire Smoke Detection in Videos. In Proceedings of the 25th International Conference on Pattern Recognition, Milan, Italy, 10–15 January 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 8523–8530. [Google Scholar]
- Zhu, G.; Chen, Z.; Liu, C.; Rong, X.; He, W. 3D video semantic segmentation for wildfire smoke. Mach. Vis. Appl. 2020, 31, 50. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; IEEE: Piscataway, NJ, USA, 2018; p. 00745. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. CBAM: Convolutional block attention module. In Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; Springer: Cham, Switzerland, 2018; pp. 8–14. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Curran Associates Inc.: Redhook, NY, USA, 2017; pp. 6000–6010. [Google Scholar]
- Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image Is Worth 16 × 16 Words: Transformers For Image Recognition At Scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
- Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv 2021, arXiv:2103.14030. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; pp. 234–241. [Google Scholar]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision, Munich, Germany, 8–14 September 2018; Springer: Berlin/Heidelberg, Germany, 2018; pp. 801–818. [Google Scholar]
Method | MRGA | GFM | mIoU | mPA | FPS |
---|---|---|---|---|---|
U-Net | 86.81 | 91.84 | 22.36 | ||
GFM–U-Net | √ | 89.40 | 94.32 | 18.95 | |
MRGA–U-Net | √ | 89.12 | 94.18 | 24.55 | |
Smoke–U-Net | √ | √ | 91.82 | 96.62 | 20.17 |
Method | Large | Medium | Small |
---|---|---|---|
SegNet | 85.34% | 82. 89% | 78.52% |
PSPNet | 86.53% | 82.30% | 80.18% |
U-Net | 89.72% | 86.92% | 83.82% |
DeepLab v3+ | 91.03% | 88.61% | 85.25% |
Smoke–U-Net | 94.97% | 92.15% | 88.36% |
Model | mIoU/% | mPA% | FPS |
---|---|---|---|
FCN | 80.26 | 85.95 | 18.28 |
SegNet | 82.32 | 88.67 | 20.65 |
PSPNet | 85.81 | 90.07 | 16.47 |
DeepLab v3+ | 88.96 | 93.20 | 19.01 |
U-Net | 86.01 | 91.84 | 22.36 |
DANet | 88.47 | 92.65 | 16.05 |
Vision Transformer | 88.24 | 92.32 | 10.30 |
Smoke–U-Net | 91.83 | 96.62 | 20.17 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zheng, Y.; Wang, Z.; Xu, B.; Niu, Y. Multi-Scale Semantic Segmentation for Fire Smoke Image Based on Global Information and U-Net. Electronics 2022, 11, 2718. https://doi.org/10.3390/electronics11172718
Zheng Y, Wang Z, Xu B, Niu Y. Multi-Scale Semantic Segmentation for Fire Smoke Image Based on Global Information and U-Net. Electronics. 2022; 11(17):2718. https://doi.org/10.3390/electronics11172718
Chicago/Turabian StyleZheng, Yuanpan, Zhenyu Wang, Boyang Xu, and Yiqing Niu. 2022. "Multi-Scale Semantic Segmentation for Fire Smoke Image Based on Global Information and U-Net" Electronics 11, no. 17: 2718. https://doi.org/10.3390/electronics11172718
APA StyleZheng, Y., Wang, Z., Xu, B., & Niu, Y. (2022). Multi-Scale Semantic Segmentation for Fire Smoke Image Based on Global Information and U-Net. Electronics, 11(17), 2718. https://doi.org/10.3390/electronics11172718