Detection of Collapsed Bridges from Multi-Temporal SAR Intensity Images by Machine Learning Techniques
Abstract
:1. Introduction
2. Data Sets of Collapsed Bridges
2.1. Satellite Images and Pre-Processing
2.2. Backscatter Model of Bridges
2.3. Extraction of Change Features
2.4. Generation of Data Sets
3. Machine Learning Models
3.1. Models Using Data Set 1
3.2. Models Using the Dataset 2
3.3. Comparison and Improvement of the Models
4. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- National Institute for Land and Infrastructure Management (NILM). Annual Report of Road-Related Research in FY 2013. Technical Note of NILIM 2013, No. 843. pp. 3–110. Available online: http://www.nilim.go.jp/lab/bcg/siryou/tnn/tnn0843.htm (accessed on 1 July 2021).
- Japan Bridge Association Inc. Investigation Report of Bridge Damages in the Kumamoto Earthquake. Available online: https://www.jasbc.or.jp/whatsnew/files/161227/kyoryohigaityousahoukokusyo.pdf (accessed on 1 July 2021). (In Japanese).
- Tanabe, R.; Yamazaki, F.; Liu, W. Damage analysis of landslides and bridges in Minami-Aso village due to 2016 Kumamoto earthquake using full-polarimetric airborne SAR images. In Proceedings of the 39th Asian Conference on Remote Sensing, Kuala Lumpur, Malaysia, 15–19 October 2018; pp. 1545–1553. [Google Scholar]
- Joyce, K.E.; Belliss, S.E.; Samsonov, S.V.; McNeill, S.J.; Glassey, P.J. A review of the status of satellite remote sensing and image processing techniques for mapping natural hazards and disasters. Prog. Phys. Geogr. Earth Environ. 2009, 33, 183–207. [Google Scholar] [CrossRef] [Green Version]
- Dell’Acqua, F.; Gamba, P. Remote Sensing and Earthquake Damage Assessment: Experiences, Limits, and Perspectives. Proc. IEEE 2012, 100, 2876–2890. [Google Scholar] [CrossRef]
- Lin, L.; Di, L.; Yu, E.G.; Kang, L.; Shrestha, R.; Rahman, S.; Tang, J.; Deng, M.; Sun, Z.; Zhang, C.; et al. A review of remote sensing in flood assessment. In Proceedings of the 2016 Fifth International Conference on Agro-Geoinformatics (Agro-Geoinformatics), Tianjin, China, 18–20 July 2016; Volume 16373607, pp. 1–4. [Google Scholar]
- Koshimura, S.; Moya, L.; Mas, E.; Bai, Y. Tsunami Damage Detection with Remote Sensing: A Review. Geosciences 2020, 10, 177. [Google Scholar] [CrossRef]
- Saito, K.; Spence, R.J.S.; Going, C.; Markus, M. Using High-Resolution Satellite Images for Post-Earthquake Building Damage Assessment: A Study following the 26 January 2001 Gujarat Earthquake. Earthq. Spectra 2004, 20, 145–169. [Google Scholar] [CrossRef]
- Miura, H.; Midorikawa, S.; Kerle, N. Detection of Building Damage Areas of the 2006 Central Java, Indonesia, Earthquake through Digital Analysis of Optical Satellite Images. Earthq. Spectra 2013, 29, 453–473. [Google Scholar] [CrossRef]
- Janalipour, M.; Mohammadzadeh, A. A Fuzzy-GA Based Decision Making System for Detecting Damaged Buildings from High-Spatial Resolution Optical Images. Remote Sens. 2017, 9, 349. [Google Scholar] [CrossRef] [Green Version]
- Adriano, B.; Xia, J.; Baier, G.; Yokoya, N.; Koshimura, S. Multi-Source Data Fusion Based on Ensemble Learning for Rapid Building Damage Mapping during the 2018 Sulawesi Earthquake and Tsunami in Palu, Indonesia. Remote Sens. 2019, 11, 886. [Google Scholar] [CrossRef] [Green Version]
- Balz, T.; Perissin, D.; Soergel, U.; Zhang, L.; Liao, M. Post-seismic infrastructure damage assessment using high-resolution SAR satellite data. Proceedings of the 2nd International Conference on Earth Observation for Global Change. 2009. Available online: https://engineering.purdue.edu/~perissin/Publish/09EOGC_hres.pdf (accessed on 1 July 2021).
- Chen, F.; Lin, H.; Li, Z.; Chen, Q.; Zhou, J. Interaction between permafrost and infrastructure along the Qinghai–Tibet Railway detected via jointly analysis of C- and L-band small baseline SAR interferometry. Remote Sens. Environ. 2012, 123, 532–540. [Google Scholar] [CrossRef]
- Lu, H.; Zhang, H.; Deng, Y.; Wang, J.; Yu, W. Building 3-D Reconstruction With a Small Data Stack Using SAR Tomography. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 2461–2474. [Google Scholar] [CrossRef]
- Balz, T.; Liao, M. Building-damage detection using post-seismic high-resolution SAR satellite data. Int. J. Remote Sens. 2010, 31, 3369–3391. [Google Scholar] [CrossRef]
- Plank, S. Rapid Damage Assessment by Means of Multi-Temporal SAR—A Comprehensive Review and Outlook to Sentinel-1. Remote Sens. 2014, 6, 4870–4906. [Google Scholar] [CrossRef] [Green Version]
- Nie, Y.; Zeng, Q.; Zhang, H.; Wang, Q. Building Damage Detection Based on OPCE Matching Algorithm Using a Single Post-Event PolSAR Data. Remote Sens. 2021, 13, 1146. [Google Scholar] [CrossRef]
- Ge, P.; Gokon, H.; Meguro, K. A review on synthetic aperture radar-based building damage assessment in disasters. Remote Sens. Environ. 2020, 240, 111693. [Google Scholar] [CrossRef]
- Adriano, B.; Yokoya, N.; Xia, J.; Miura, H.; Liu, W.; Matsuoka, M.; Koshimura, S. Learning from multimodal and multitemporal earth observation data for building damage mapping. ISPRS J. Photogramm. Remote Sens. 2021, 175, 132–143. [Google Scholar] [CrossRef]
- Abdi, G.; Jabari, S. A Multi-Feature Fusion Using Deep Transfer Learning for Earthquake Building Damage Detection. Can. J. Remote Sens. 2021, 47, 1–16. [Google Scholar] [CrossRef]
- Wang, Y.; Zheng, Q. Recognition of roads and bridges in SAR images. Pattern Recognit. 1998, 31, 953–962. [Google Scholar] [CrossRef]
- Soergel, U.; Cadario, E.; Thiele, A.; Thoennessen, U. Extraction of bridges over water in multi-aspect high-resolution InSAR data. In Proceedings of the Symposium of ISPRS Comission III Photogrametric Computer Vision PCV’ 06, Bonn, Germany, 20–22 September 2006; Available online: https://www.isprs.org/proceedings/XXXVI/part3/singlepapers/P_17.pdf (accessed on 1 July 2021).
- Liu, W.; Sawa, K.; Yamazaki, F. Backscattering characteristics of bridges from high-resolution X-band SAR imagery. In Proceedings of the International Symposium on Remote Sensing 2017, Nagoya, Japan, 17–19 May 2017; pp. 324–327. [Google Scholar]
- Cadario, E.; Gross, H.; Hammer, H.; Schulz, K.; Thiele, A.; Thoennessen, U.; Soergel, U.; Weydahl, D.J. Change Detection for Bridges over Water in Airborne and Spaceborne SAR Data. In Proceedings of the IGARSS 2008 IEEE International Geoscience and Remote Sensing Symposium, Boston, MA, USA, 7–11 July 2008; Volume 5, pp. 479–482. [Google Scholar] [CrossRef]
- Yang, M.; Jian, Z. Bridge detection in high-resolution X-band SAR images by combined statistical and topological approach. Prog. Electromagn. Res. M 2017, 57, 91–102. [Google Scholar] [CrossRef] [Green Version]
- Chen, L.; Weng, T.; Xing, J.; Pan, Z.; Yuan, Z.; Xing, X.; Zhang, P. A New Deep Learning Network for Automatic Bridge Detection from SAR Images Based on Balanced and Attention Mechanism. Remote Sens. 2020, 12, 441. [Google Scholar] [CrossRef]
- Jiang, K.; Wang, C.; Zhang, H.; Chen, W.; Zhang, B.; Tang, Y.; Wu, F. Damage analysis of 2008 Wenchuan earthquake using SAR images. In Proceedings of the 2009 IEEE International Geoscience and Remote Sensing Symposium, Cape Town, South Africa, 12–17 July 2019. [Google Scholar] [CrossRef]
- Inoue, K.; Liu, W.; Yamazaki, F. Detection of Bridge Damages due to Tsunami Using Multi-temporal High-resolution SAR Images. J. Jpn. Assoc. Earthq. Eng. 2017, 17, 5. [Google Scholar] [CrossRef] [Green Version]
- Liu, W.; Yamazaki, F. Extraction of Collapsed Bridges Due to the 2011 Tohoku-Oki Earthquake from Post-Event SAR Images. J. Disaster Res. 2018, 13, 281–290. [Google Scholar] [CrossRef]
- Liu, W.; Yamazaki, F. Bridge damage assessment using single post-event TerraSAR-X image. In Proceedings of the IEEE 2019 International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 28 July–2 August 2019; pp. 4833–4836. [Google Scholar] [CrossRef]
- Sester, M.; Feng, Y.; Thiemann, F. Building generalization using deep learning. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2018, XLII-4, 565–572. [Google Scholar] [CrossRef] [Green Version]
- Zhang, X.; Liu, G.; Zhang, C.; Atkinson, P.M.; Tan, X.; Jian, X.; Zhou, X.; Li, Y. Two-Phase Object-Based Deep Learning for Multi-Temporal SAR Image Change Detection. Remote Sens. 2020, 12, 548. [Google Scholar] [CrossRef] [Green Version]
- Derksen, D.; Inglada, J.; Michel, J. Geometry Aware Evaluation of Handcrafted Superpixel-Based Features and Convolutional Neural Networks for Land Cover Mapping Using Satellite Imagery. Remote Sens. 2020, 12, 513. [Google Scholar] [CrossRef] [Green Version]
- Belgiu, M.; Drăguţ, L. Random forest in remote sensing: A review of applications and future directions. ISPRS J. Photogramm. Remote Sens. 2016, 114, 24–31. [Google Scholar] [CrossRef]
- Bioucas-Dias, J.M.; Plaza, A.; Camps-Valls, G.; Scheunders, P.; Nasrabadi, N.; Chanussot, J. Hyperspectral Remote Sensing Data Analysis and Future Challenges. IEEE Geosci. Remote Sens. Mag. 2013, 1, 6–36. [Google Scholar] [CrossRef] [Green Version]
- Ray, S. A quick review of machine learning algorithms. In Proceedings of the 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (Com-IT-Con), Faridabad, India, 14–16 February 2019; pp. 35–39. [Google Scholar] [CrossRef]
- Shoji, G.; Nakamura, T. Damage Assessment of Road Bridges Subjected to the 2011 Tohoku Pacific Earthquake Tsunami. J. Disaster Res. 2017, 12, 79–89. [Google Scholar] [CrossRef]
- Japan Weather Association. Characteristics of rainfall in Kumamoto Heavy Rain and Future Outlook (Flash Report). Disaster Report. 2020. Available online: https://www.jwa.or.jp/news/2020/07/10378/ (accessed on 1 July 2021). (In Japanese).
- Cabinet Office, Government of Japan. Report of the Damage Situation Related to 14:00 on 2 November 2020. Available online: http://www.bousai.go.jp/updates/r2_07ooame/pdf/r20703_ooame_38.pdf (accessed on 1 July 2021). (In Japanese)
- Liu, W.; Maruyama, Y.; Yamazaki, F. Damage assessment of bridges due to the 2020 July Flood in Japan using ALOS-2 intensity images. In Proceedings of the IEEE 2021 International Geoscience and Remote Sensing Symposium, Online, 12–16 July 2021. [Google Scholar]
- Hirano, H.; Yamazaki, F.; Liu, W. Backscattering characteristics of bridges from airborne full-polarimetric SAR images. In Proceedings of the 38th Asian Conference on Remote Sensing, New Delhi, India, 23–27 October 2017. [Google Scholar]
- National Institute of Information and Communications Technology (NICT). Available online: https://www.nict.go.jp/out-promotion/other/case-studies/itenweb/Pi-sar2.html (accessed on 1 July 2021). (In Japanese).
- Geospatial Information Authority of Japan (GSI). Digital Map of Japan. Available online: https://maps.gsi.go.jp (accessed on 1 July 2021). (In Japanese)
- Geospatial Information Authority of Japan (GSI). Fundamental Geospatial Data. Available online: https://www.gsi.go.jp/kiban/ (accessed on 1 July 2021). (In Japanese)
- Liu, W.; Yamazaki, F. Detection of Crustal Movement From TerraSAR-X Intensity Images for the 2011 Tohoku, Japan Earthquake. IEEE Geosci. Remote Sens. Lett. 2012, 10, 199–203. [Google Scholar] [CrossRef]
- Liu, W.; Yamazaki, F.; Matsuoka, M.; Nonaka, T.; Sasagawa, T. Estimation of three-dimensional crustal movements in the 2011 Tohoku-Oki, Japan, earthquake from TerraSAR-X intensity images. Nat. Hazards Earth Syst. Sci. 2015, 15, 637–645. [Google Scholar] [CrossRef] [Green Version]
- Liu, W.; Yamazaki, F. Review article: Detection of inundation areas due to the 2015 Kanto and Tohoku torrential rain in Japan based on multi-temporal ALOS-2 imagery. Nat. Hazards Earth Syst. Sci. 2018, 18, 1905–1918. [Google Scholar] [CrossRef] [Green Version]
- Liu, W.; Fujii, K.; Maruyama, Y.; Yamazaki, F. Inundation Assessment of the 2019 Typhoon Hagibis in Japan Using Multi-Temporal Sentinel-1 Intensity Images. Remote Sens. 2021, 13, 639. [Google Scholar] [CrossRef]
- Liaw, A.; Wiener, M. Classification and Regression by randomForest. R News 2002, 2, 18–22. [Google Scholar]
- Scikit-Learn Developers. Scikit-Learn. Retrieved October 2020. 2020. Available online: https://scikit-learn.org/stable/index.html (accessed on 1 July 2021).
- Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res. 2002, 16, 321–357. [Google Scholar] [CrossRef]
- Imbalanced-Learn Documentation. Available online: https://imbalanced-learn.org/stable/# (accessed on 1 July 2021).
No. | Damage Condition | No. | Damage Condition | No. | Damage Condition |
---|---|---|---|---|---|
M159 | a | K12 | a | K27 | b |
M160 | a | K16 | b | K28 | b |
M163 | a | K18 | a | K29 | b |
M164 | c | K19 | b | K30 | a |
M165 | c | K21 | b | K32 | a |
M166 | c | K23 | a | K40 | b |
M168 | c | K25 | b | K44 | b |
Symbol | Equations |
---|---|
where is the thresholding value for significant changes |
Condition | Data Set 1 | Data Set 2 | ||||
---|---|---|---|---|---|---|
July Floods | Tohoku EarthQuake | Mixed Events | ||||
Training Set | Test Set | Validation Set | Training Set | Test Set | Validation Set | |
Collapsed | 11 | 3 | 7 | 10 | 5 | 6 |
Surviving | 19 | 11 | 37 | 38 | 8 | 21 |
Total | 30 | 14 | 44 | 48 | 13 | 27 |
Features | Importance of RF Model | |||
5 Features | 4 Features | 3 Features | ||
Correlation () | 0.31 | 0.24 | 0.33 | |
Average of difference () | 0.25 | 0.46 | 0.41 | |
STD of difference () | 0.12 | 0.12 | - | |
Minimum of difference () | 0.20 | 0.17 | 0.26 | |
Percentage of significantly changed area () | 0.11 | - | - | |
Value | ||||
Hyperparameters | n_estimator | 5 | 10 | 30 |
max_depth | 5 | 3 | 3 | |
random_state | 42 | 7 | 42 | |
Training set (30 bridges) | Recall | 0.91 | 1.00 | 0.91 |
Precision | 1.00 | 1.00 | 1.00 | |
Accuracy | 0.97 | 1.00 | 0.97 | |
Kappa | 0.93 | 1.00 | 0.93 | |
Test set (14 bridges) | Recall | 0.67 | 0.67 | 0.67 |
Precision | 1.00 | 1.00 | 1.00 | |
Accuracy | 0.93 | 0.93 | 0.93 | |
Kappa | 0.76 | 0.76 | 0.76 |
44 Bridges | Ground Truth | Precision | |||
---|---|---|---|---|---|
Collapsed | Surviving | Total | |||
Prediction | Collapsed | 5 | 12 | 17 | 0.29 |
Surviving | 2 | 25 | 27 | 0.93 | |
Total | 7 | 37 | 44 | ||
Recall | 0.71 | 0.68 | Accuracy | 0.68 |
Features | Coefficient of LR Model | |||
5 Features | 4 Features | 3 Features | ||
Correlation () | 0.08 | −0.42 | −0.75 | |
Average of difference () | 0.65 | 1.55 | 1.49 | |
STD of difference () | −0.16 | 0.31 | - | |
Minimum of difference () | −0.05 | - | - | |
Percentage of significantly changed area () | −0.04 | 0.52 | 0.76 | |
Intercept | 1.55 | 2.61 | 2.59 | |
Value | ||||
Hyperparameters | C | 0.1 | 10 | 10 |
random_state | 0 | 0 | 0 | |
Training set (30 bridges) | Recall | 0.64 | 0.82 | 0.82 |
Precision | 1.00 | 0.90 | 0.90 | |
Accuracy | 0.87 | 0.90 | 0.90 | |
Kappa | 0.69 | 0.78 | 0.78 | |
Test set (14 bridges) | Recall | 0.67 | 1.00 | 1.00 |
Precision | 1.00 | 0.75 | 0.75 | |
Accuracy | 0.93 | 0.93 | 0.93 | |
Kappa | 0.76 | 0.81 | 0.81 |
44 Bridges | Ground Truth | Precision | ||
---|---|---|---|---|
Collapsed | Surviving | Total | ||
Collapsed | 5 | 10 | 15 | 0.33 |
Surviving | 2 | 27 | 29 | 0.93 |
Total | 7 | 37 | 44 | |
0.71 | 0.73 | Accuracy | 0.73 |
Features | Importance of RF Model | |||
5 Features | 4 Features | 3 Features | ||
Correlation () | 0.17 | 0.27 | 0.28 | |
Average of difference () | 0.09 | 0.11 | − | |
STD of difference () | 0.08 | − | − | |
Minimum of difference () | 0.40 | 0.39 | 0.39 | |
Percentage of significantly changed area () | 0.26 | 0.34 | 0.34 | |
Value | ||||
Parameters | n_estimator | 10 | 5 | 10 |
max_depth | 3 | 3 | 3 | |
random_state | 0 | 7 | 7 | |
Training set (48 bridges) | Recall | 0.70 | 0.70 | 0.60 |
Precision | 1.00 | 1.00 | 1.00 | |
Accuracy | 0.94 | 0.94 | 0.92 | |
Kappa | 0.79 | 0.79 | 0.70 | |
Test set (13 bridges) | Recall | 0.60 | 0.40 | 0.40 |
Precision | 1.00 | 1.00 | 1.00 | |
Accuracy | 0.85 | 0.77 | 0.77 | |
Kappa | 0.65 | 0.45 | 0.45 | |
Validation set (27 bridges) | Recall | 0.83 | 0.67 | 0.83 |
Precision | 0.83 | 0.80 | 0.83 | |
Accuracy | 0.93 | 0.89 | 0.93 | |
Kappa | 0.79 | 0.66 | 0.79 |
Features | Coefficient of LR Model | |||
5 Features | 4 Features | 3 Features | ||
Correlation () | 0.07 | 1.05 | 0.827 | |
Average of difference () | 0.30 | 0.34 | 0.311 | |
STD of difference () | −0.002 | − | − | |
Minimum of difference () | 0.07 | −0.03 | − | |
Percentage of significantly changed area () | −0.06 | −8.60 | −7.97 | |
Intercept | 3.34 | 1.84 | 2.44 | |
Value | ||||
Parameters | C | 100 | 100 | 10 |
random_state | 0 | 0 | 0 | |
Training set (48 bridges) | Recall | 0.54 | 0.54 | 0.54 |
Precision | 1.00 | 1.00 | 1.00 | |
Accuracy | 0.90 | 0.90 | 0.90 | |
Kappa | 0.61 | 0.61 | 0.61 | |
Test set (13 bridges) | Recall | 0.60 | 0.60 | 0.60 |
Precision | 1.00 | 1.00 | 1.00 | |
Accuracy | 0.85 | 0.85 | 0.85 | |
Kappa | 0.65 | 0.65 | 0.65 | |
Validation set (27 bridges) | Recall | 0.67 | 0.67 | 0.67 |
Precision | 0.80 | 0.80 | 0.80 | |
Accuracy | 0.89 | 0.89 | 0.89 | |
Kappa | 0.66 | 0.66 | 0.66 |
Features | Importance of RF Model | |
---|---|---|
Correlation () | 0.08 | |
Average of difference () | 0.28 | |
STD of difference () | 0.01 | |
Minimum of difference () | 0.40 | |
Percentage of significantly changed area () | 0.23 | |
Value | ||
Parameters | n_estimator | 5 |
max_depth | 3 | |
random_state | 0 | |
Training (48 bridges) | Recall | 0.80 |
Precision | 0.80 | |
Accuracy | 0.92 | |
Kappa | 0.75 | |
Test (13 bridges) | Recall | 0.80 |
Precision | 0.80 | |
Accuracy | 0.85 | |
Kappa | 0.68 | |
Verification (27 bridges) | Recall | 1.00 |
Precision | 0.75 | |
Accuracy | 0.93 | |
Kappa | 0.81 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, W.; Maruyama, Y.; Yamazaki, F. Detection of Collapsed Bridges from Multi-Temporal SAR Intensity Images by Machine Learning Techniques. Remote Sens. 2021, 13, 3508. https://doi.org/10.3390/rs13173508
Liu W, Maruyama Y, Yamazaki F. Detection of Collapsed Bridges from Multi-Temporal SAR Intensity Images by Machine Learning Techniques. Remote Sensing. 2021; 13(17):3508. https://doi.org/10.3390/rs13173508
Chicago/Turabian StyleLiu, Wen, Yoshihisa Maruyama, and Fumio Yamazaki. 2021. "Detection of Collapsed Bridges from Multi-Temporal SAR Intensity Images by Machine Learning Techniques" Remote Sensing 13, no. 17: 3508. https://doi.org/10.3390/rs13173508
APA StyleLiu, W., Maruyama, Y., & Yamazaki, F. (2021). Detection of Collapsed Bridges from Multi-Temporal SAR Intensity Images by Machine Learning Techniques. Remote Sensing, 13(17), 3508. https://doi.org/10.3390/rs13173508