Mapping of Dwellings in IDP/Refugee Settlements from Very High-Resolution Satellite Imagery Using a Mask Region-Based Convolutional Neural Network
Abstract
:1. Introduction
- Transfer learning through domain adaptation [40]: Given that deep learning models are data driven, can we gain leverage from openly available pretrained weights generated from non-remotely sensed datasets through domain adaptation for dwelling detection? Which datasets perform better with regard to the provision of weights? Is there a pronounced performance difference if we train the model from scratch?
- Temporal transferability: Given that camps are very dynamic (Figure 1), is it possible to extract dwelling features using a model trained on images obtained in the past, every time new imagery is obtained?
2. Materials and Methods
2.1. Study Site
2.2. Data, Processing, and Sample Preparation
2.3. Model and Training Procedure
3. Results
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
References
- UNHCR Refugee Data Finder. Available online: https://www.unhcr.org/refugee-statistics/ (accessed on 15 June 2021).
- Kemper, T.; Heinzel, J. Mapping and monitoring of refugees and internally displaced people using EO data. In Global Urban Monitoring and Assessment: Through Earth Observation; Weng, Q., Ed.; CRC Press Taylor & Francis Group: Boca Raton, FL, USA, 2014; pp. 195–216. ISBN 9781466564503. [Google Scholar]
- Kranz, O.; Zeug, G.; Tiede, D.; Clandillon, S.; Bruckert, D.; Kemper, T.; Lang, S.; Caspard, M. Monitoring Refugee/IDP camps to Support International Relief Action. In Geoinformation for Disaster and Risk Management–Examples and Best Practices; Altan, O., Backhaus, R., Boccardo, P., Zlatanova, S., Eds.; Joint Board of Geospatial Information Societies (JB GIS): Copenhagen, Denmark, 2010; pp. 51–56. ISBN 978-87-90907-88-4. [Google Scholar]
- Wang, S.; So, E.; Smith, P. Detecting tents to estimate the displaced populations for post-disaster relief using high resolution satellite imagery. Int. J. Appl. Earth Obs. Geoinf. 2015, 36, 87–93. [Google Scholar] [CrossRef]
- Lang, S.; Füreder, P.; Riedler, B.; Wendt, L.; Braun, A.; Tiede, D.; Schoepfer, E.; Zeil, P.; Spröhnle, K.; Kulessa, K.; et al. Earth observation tools and services to increase the effectiveness of humanitarian assistance. Eur. J. Remote Sens. 2020, 53, 67–85. [Google Scholar] [CrossRef] [Green Version]
- Jenerowicz, M.; Wawrzaszek, A.; Drzewiecki, W.; Krupinski, M.; Aleksandrowicz, S. Multifractality in Humanitarian Applications: A Case Study of Internally Displaced Persons/Refugee Camps. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 4438–4445. [Google Scholar] [CrossRef]
- Tomaszewski, B.; Tibbets, S.; Hamad, Y.; Al-Najdawi, N. Infrastructure evolution analysis via remote sensing in an urban refugee camp–Evidence from Za’atari. Procedia Eng. 2016, 159, 118–123. [Google Scholar] [CrossRef]
- Braun, A.; Lang, S.; Hochschild, V. Impact of Refugee Camps on Their Environment A Case Study Using Multi-Temporal SAR Data. J. Geogr. Environ. Earth Sci. Int. 2016, 4, 1–17. [Google Scholar] [CrossRef]
- Braun, A.; Fakhri, F.; Hochschild, V. Refugee camp monitoring and environmental change assessment of Kutupalong, Bangladesh, based on radar imagery of Sentinel-1 and ALOS-2. Remote Sens. 2019, 11, 2047. [Google Scholar] [CrossRef] [Green Version]
- Leiterer, R.; Bloesch, U.; Wulf, H.; Eugster, S.; Joerg, P.C. Vegetation monitoring in refugee-hosting areas in South Sudan. Appl. Geogr. 2018, 93, 1–15. [Google Scholar] [CrossRef]
- Hagenlocher, M.; Lang, S.; Tiede, D. Integrated assessment of the environmental impact of an IDP camp in Sudan based on very high resolution multi-temporal satellite imagery. Remote Sens. Environ. 2012, 126, 27–38. [Google Scholar] [CrossRef]
- Benz, S.; Park, H.; Li, J.; Crawl, D.; Block, J.; Nguyen, M.; Altintas, I. Understanding a rapidly expanding refugee camp using convolutional neural networks and satellite imagery. In Proceedings of the IEEE 15th International Conference on eScience (eScience), San Diego, CA, USA, 24–27 September 2019. [Google Scholar]
- Lang, S.; Tiede, D.; Hölbling, D.; Füreder, P.; Zeil, P. Earth observation (EO)-based ex post assessment of internally displaced person (IDP) camp evolution and population dynamics in Zam Zam, Darfur. Int. J. Remote Sens. 2010, 31, 5709–5731. [Google Scholar] [CrossRef]
- Lang, S.; Tiede, D.; Hofer, F. Modeling ephemeral settlements using VHSR image data and 3D visualization–The example of Goz Amer refugee camp in Chad. PFG-Photogramm. Fernerkundung, Geoinf. 2006, 4, 327–337. [Google Scholar]
- Tiede, D.; Füreder, P.; Lang, S.; Hölbling, D.; Zeil, P. Automated analysis of satellite imagery to provide information products for humanitarian relief operations in refugee camps -from scientific development towards operational services. Photogramm. Fernerkundung Geoinf. 2013, 2013, 185–195. [Google Scholar] [CrossRef]
- Ahmed, N.; Firoze, A.; Rahman, R.M. Machine learning for predicting landslide risk of rohingya refugee camp infrastructure. J. Inf. Telecommun. 2020, 4, 175–198. [Google Scholar] [CrossRef] [Green Version]
- Hadzic, A.; Christie, G.; Freeman, J.; Dismer, A.; Bullard, S.; Greiner, A.; Jacobs, N.; Mukherjee, R. Estimating Displaced Populations from Overhead. In Proceedings of the International Geoscience and Remote Sensing Symposium (IGARSS), Waikoloa, HI, USA, 26 September–2 October 2020. [Google Scholar]
- Ahmed, N.; DIptu, N.A.; Shadhin, M.S.K.; Jaki, M.A.F.; Hasan, M.F.; Islam, M.N.; Rahman, R.M. Artificial Neural Network and Machine Learning Based Methods for Population Estimation of Rohingya Refugees: Comparing Data-Driven and Satellite Image-Driven Approaches. Vietnam J. Comput. Sci. 2019, 6, 439–455. [Google Scholar] [CrossRef] [Green Version]
- Green, B.; Blanford, J.I. Refugee camp population estimates using automated feature extraction. In Proceedings of the Annual Hawaii International Conference on System Sciences, Big Island, HI, USA, 10 January 2002. [Google Scholar]
- Spröhnle, K.; Tiede, D.; Schoepfer, E.; Füreder, P.; Svanberg, A.; Rost, T. Earth observation-based dwelling detection approaches in a highly complex refugee camp environment–A comparative study. Remote Sens. 2014, 6, 9277–9297. [Google Scholar] [CrossRef] [Green Version]
- Kahraman, F.; Ates, H.F.; Kucur Ergunay, S.S. Automated Detection Of Refugee/IDP TENTS FROM Satellite Imagery Using Two- Level Graph Cut Segmentation. In Proceedings of the CaGIS/ASPRS Fall Conference, San Antonio, TX, USA, 27–30 October 2013. [Google Scholar]
- Sprohnle, K.; Fuchs, E.M.; Aravena Pelizari, P. Object-Based Analysis and Fusion of Optical and SAR Satellite Data for Dwelling Detection in Refugee Camps. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 1780–1791. [Google Scholar] [CrossRef] [Green Version]
- Tiede, D.; Lang, S.; Hölbling, D.; Füreder, P. Transferability of obia rulesets for idp camp analysis in darfur. In Proceedings of the GEOBIA 2010: Geographic Object-Based Image Analysis, Ghent, Belgium, 29 June–2 July 2010; Addink, E.A., van Coillie, F.M.B., Eds.; ISPRS: Ghent, Belgium, 2010; XXXVIII-4/C7. [Google Scholar]
- Aravena Pelizari, P.; Spröhnle, K.; Geiß, C.; Schoepfer, E.; Plank, S.; Taubenböck, H. Multi-sensor feature fusion for very high spatial resolution built-up area extraction in temporary settlements. Remote Sens. Environ. 2018, 209, 793–807. [Google Scholar] [CrossRef]
- UNHCR Bangladesh Refugee Emergency: Population Factsheet (as of 30 September 2019). Available online: https://reliefweb.int/sites/reliefweb.int/files/resources/68229.pdf (accessed on 15 May 2021).
- Alzubaidi, L.; Zhang, J.; Humaidi, A.J.; Al-Dujaili, A.; Duan, Y.; Al-Shamma, O.; Santamaría, J.; Fadhel, M.A.; Al-Amidie, M.; Farhan, L. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data 2021, 8. [Google Scholar] [CrossRef]
- Lu, Z.; Xu, T.; Liu, K.; Liu, Z.; Zhou, F.; Liu, Q. 5M-Building: A large-scale high-resolution building dataset with CNN based detection analysis. In Proceedings of the International Conference on Tools with Artificial Intelligence, ICTAI, Portland, OR, USA, 4–6 November 2019. [Google Scholar]
- Tiede, D.; Schwendemann, G.; Alobaidi, A.; Wendt, L.; Lang, S. Mask R-CNN-based building extraction from VHR satellite data in operational humanitarian action: An example related to Covid-19 response in Khartoum, Sudan. Trans. GIS 2021, 25, 1213–1227. [Google Scholar] [CrossRef]
- Pasquali, G.; Iannelli, G.C.; Dell’Acqua, F. Building footprint extraction from multispectral, spaceborne earth observation datasets using a structurally optimized U-Net convolutional neural network. Remote Sens. 2019, 11, 2803. [Google Scholar] [CrossRef] [Green Version]
- Zhu, Q.; Liao, C.; Hu, H.; Mei, X.; Li, H. MAP-Net: Multiple Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery. IEEE Trans. Geosci. Remote Sens. 2021, 59, 6169–6181. [Google Scholar] [CrossRef]
- Sirko, W.; Kashubin, S.; Ritter, M.; Annkah, A.; Bouchareb, Y.S.E.; Dauphin, Y.; Keysers, D.; Neumann, M.; Cisse, M.; Quinn, J. Continental-Scale Building Detection from High Resolution Satellite Imagery. arXiv 2021, arXiv:2107.12283. [Google Scholar]
- Tiecke, T.G.; Liu, X.; Zhang, A.; Gros, A.; Li, N.; Yetman, G.; Kilic, T.; Murray, S.; Blankespoor, B.; Prydz, E.B.; et al. Mapping the world population one building at a time. arXiv 2017, arXiv:1712.05839. [Google Scholar] [CrossRef]
- Van Den Hoek, J.; Friedrich, H.K. Satellite-based human settlement datasets inadequately detect refugee settlements: A critical assessment at thirty refugee settlements in uganda. Remote Sens. 2021, 13, 3574. [Google Scholar] [CrossRef]
- Van Den Hoek, J.; Friedrich, H.K.; Ballasiotes, A.; Peters, L.E.R.; Wrathall, D. Development after displacement: Evaluating the utility of openstreetmap data for monitoring sustainable development goal progress in refugee settlements. ISPRS Int. J. Geo-Inf. 2021, 10, 153. [Google Scholar] [CrossRef]
- UNHCR SITE MAP Minawao/Gawar Cameroon. Available online: https://im.unhcr.org/apps/campmapping/?site=CMRs004589 (accessed on 20 March 2021).
- Ghorbanzadeh, O.; Tiede, D.; Wendt, L.; Sudmanns, M.; Lang, S. Transferable instance segmentation of dwellings in a refugee camp-integrating CNN and OBIA. Eur. J. Remote Sens. 2021. [Google Scholar] [CrossRef]
- Lu, Y.; Koperski, K.; Kwan, C.; Li, J. Deep Learning for Effective Refugee Tent Extraction near Syria-Jordan Border. IEEE Geosci. Remote Sens. Lett. 2021, 18, 1342–1346. [Google Scholar] [CrossRef]
- Quinn, J.A.; Nyhan, M.M.; Navarro, C.; Coluccia, D.; Bromley, L.; Luengo-Oroz, M. Humanitarian applications of machine learning with remote-sensing data: Review and case study in refugee settlement mapping. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 2018, 376. [Google Scholar] [CrossRef] [Green Version]
- Ghorbanzadeh, O.; Tiede, D.; Dabiri, Z.; Sudmanns, M.; Lang, S. Dwelling extraction in refugee camps using CNN-First experiences and lessons learnt. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.-ISPRS Arch. 2018, 42, 161–166. [Google Scholar] [CrossRef] [Green Version]
- Long, M.; Wang, J.; Cao, Y.; Sun, J.; Yu, P.S. Deep learning of transferable representation for scalable domain adaptation. IEEE Trans. Knowl. Data Eng. 2016, 28, 2027–2040. [Google Scholar] [CrossRef]
- UNHCR Cameroun–Extrême Nord Profil de la Population Refugiée du Camp de Minawao Cameroun–Extrême Nord Profil de la Population Refugiée du Camp de Minawao. Available online: https://www.refworld.org/pdfid/553f384e4.pdf (accessed on 12 April 2021).
- UNHCR Shelter Design Catalgue; Shelter and Settlement Section Division of Programme Support and Management United Nations High Commissioner for Refugees: Geneva, Switzerland, 2016.
- HHI Satellite Imagery Interpretation Guide Intentional Burning of Tukuls. Available online: https://hhi.harvard.edu/files/humanitarianinitiative/files/siig_ii_burned_tukuls_3.pdf?m=1610658910 (accessed on 3 April 2021).
- World Bank Climate Change Knowledge Portal: For Developmet Practionars and Policy Makers. Available online: https://climateknowledgeportal.worldbank.org/country/cameroon/climate-data-historical (accessed on 12 February 2021).
- Digital Globe DigitalGlobe Core Imagery Products Guide. Available online: https://www.geosoluciones.cl/documentos/worldview/DigitalGlobe-Core-Imagery-Products-Guide.pdf (accessed on 17 March 2021).
- Laben, C.A.; Brower, B.V. Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening. U.S. Patent 6,011,875, 4 January 2000. [Google Scholar]
- USGS Surface Radar Topographic Mission (SRTM) Digital Elevation Model. Available online: https://earthexplorer.usgs.gov/ (accessed on 15 September 2021).
- Sabottke, C.F.; Spieler, B.M. The Effect of Image Resolution on Deep Learning in Radiography. Radiol. Artif. Intell. 2020, 2, e190015. [Google Scholar] [CrossRef] [PubMed]
- Rukundo, O. Effects of Image Size on Deep Learning. Available online: http://arxiv.org/abs/2101.11508 (accessed on 25 March 2021).
- Pawluszek-Filipiak, K.; Borkowski, A. On the importance of train-test split ratio of datasets in automatic landslide detection by supervised classification. Remote Sens. 2020, 12, 3054. [Google Scholar] [CrossRef]
- Nguyen, Q.H.; Ly, H.B.; Ho, L.S.; Al-Ansari, N.; Van Le, H.; Tran, V.Q.; Prakash, I.; Pham, B.T. Influence of data splitting on performance of machine learning models in prediction of shear strength of soil. Math. Probl. Eng. 2021, 2021. [Google Scholar] [CrossRef]
- Kahloot, K.M.; Ekler, P. Algorithmic Splitting: A Method for Dataset Preparation. IEEE Access 2021, 9, 125229–125237. [Google Scholar] [CrossRef]
- Xu, Y.; Goodacre, R. On Splitting Training and Validation Set: A Comparative Study of Cross-Validation, Bootstrap and Systematic Sampling for Estimating the Generalization Performance of Supervised Learning. J. Anal. Test. 2018, 2, 249–262. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lin, T.-Y.; Maire, M.; Belongie, S.; Bourdev, L.; Girshick, R.; Hays, J.; Perona, P.; Ramanan, D.; Zitnick, C.L.; Dollár, P. Microsoft COCO: Common Objects in Context. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2015. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Francois, C.; Joe, Y.-H.; De Thibault Main, B. Keras Code and Weights Files for Popular Deep Learning Models. Available online: https://github.com/fchollet/deep-learning-models (accessed on 1 January 2021).
- Waleed, A. Mask R-CNN for Object Detection and Instance Segmentation on Keras and TensorFlow. Available online: https://github.com/matterport/Mask_RCNN (accessed on 7 January 2021).
- He, K.; Gkioxari, G.; Dollar, P.; Girshick, R. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22 October–29 October 2017. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Girshick, R. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 13–16 December 2015; pp. 1440–1448. [Google Scholar]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. SSD: Single Shot Multibox Detector. Available online: https://arxiv.org/abs/1512.02325 (accessed on 21 January 2021).
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer: Cham, Switzerland, 2015; Volume 9351, pp. 234–241. [Google Scholar]
- Hao, S.; Zhou, Y.; Guo, Y. A Brief Survey on Semantic Segmentation with Deep Learning. Neurocomputing 2020. [Google Scholar] [CrossRef]
- Zhang, K.; Han, Y.; Chen, J.; Zhang, Z.; Wang, S. Semantic Segmentation for Remote Sensing based on RGB Images and Lidar Data using Model-Agnostic Meta-Learning and Partical Swarm Optimization. IFAC-Pap. 2020, 53, 397–402. [Google Scholar] [CrossRef]
- Ruder, S. An Overview of Gradient Descent Optimization Algorithms. Available online: http://arxiv.org/abs/1609.04747 (accessed on 3 January 2021).
- Soekhoe, D.; Van der Putten, P.; Plaat, A. On the impact of data set size in transfer learning using deep neural networks. In Proceedings of the Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Redondo Beach, CA, USA, 8–11 July 2007; Volume 9897 LNCS, pp. 50–60. [Google Scholar]
- Ghafoorian, M.; Mehrtash, A.; Kapur, T.; Karssemeijer, N.; Marchiori, E.; Pesteie, M.; Guttmann, C.R.G.; de Leeuw, F.E.; Tempany, C.M.; van Ginneken, B.; et al. Transfer learning for domain adaptation in MRI: Application in brain lesion segmentation. In Proceedings of the Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 20th International Conference, Quebec City, QC, Canada, 11–13 September 2017; Volume 10435 LNCS, pp. 516–524. [Google Scholar]
- Bodla, N.; Singh, B.; Chellappa, R.; Davis, L.S. Soft-NMS–Improving Object Detection with One Line of Code. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017. [Google Scholar]
- Sebastien, O. Building Segmentation on Satellite Images. Available online: http://arxiv.org/abs/1703.06870 (accessed on 27 April 2021).
- Zhao, K.; Kang, J.; Jung, J.; Sohn, G. Building extraction from satellite images using mask R-CNN with building boundary regularization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA, 18–22 June 2018; pp. 242–246. [Google Scholar]
- Guyon, I.; Bennett, K.; Cawley, G.; Escalante, H.J.; Escalera, S.; Ho, T.K.; Macià, N.; Ray, B.; Saeed, M.; Statnikov, A.; et al. Design of the 2015 ChaLearn AutoML challenge. In Proceedings of the International Joint Conference on Neural Networks, Killarney, Ireland, 12–17 July 2015. [Google Scholar]
- Taha, A.A.; Hanbury, A. Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool. BMC Med. Imaging 2015, 15. [Google Scholar] [CrossRef] [Green Version]
- Vakalopoulou, M.; Karantzalos, K.; Komodakis, N.; Paragios, N. Building detection in very high resolution multispectral data with deep learning features. In Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy, 26–31 July 2015; pp. 1873–1876. [Google Scholar]
- Hui, Z.; Li, Z.; Cheng, P.; Ziggah, Y.Y.; Fan, J. Building extraction from airborne lidar data based on multi-constraints graph segmentation. Remote Sens. 2021, 13, 3766. [Google Scholar] [CrossRef]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
- Yosinski, J.; Clune, J.; Bengio, Y.; Lipson, H. How transferable are features in deep neural networks? In Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; Volume 4, pp. 3320–3328. [Google Scholar]
- Wen, Y.; Chen, L.; Deng, Y.; Zhou, C. Rethinking pre-training on medical imaging. J. Vis. Commun. Image Represent. 2021, 78. [Google Scholar] [CrossRef]
- He, K.; Girshick, R.; Dollar, P. Rethinking imageNet pre-training. In Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea, 27 October–2 November 2019; pp. 4917–4926. [Google Scholar]
- Zhao, K.; Kang, J.; Jung, J.; Sohn, G. Building extraction from satellite images using mask R-CNN with building boundary regularization. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work. 2018, 2018, 242–246. [Google Scholar] [CrossRef]
- Hu, Y.; Guo, F. Building Extraction Using Mask Scoring R-CNN Network. In Proceedings of the 3rd International Conference on Computer Science and Application Engineering, Sanya, China, 22–24 October 2019; pp. 1–5. [Google Scholar]
- Wen, Q.; Jiang, K.; Wang, W.; Liu, Q.; Guo, Q.; Li, L.; Wang, P. Automatic building extraction from google earth images under complex backgrounds based on deep instance segmentation network. Sensors 2019, 19, 333. [Google Scholar] [CrossRef] [Green Version]
- Kemper, T.; Pesaresi, M.; Soille, P.; Jenerowicz, M. Enumeration of Dwellings in Darfur Camps from GeoEye-1 Satellite Images Using Mathematical Morphology. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2011, 4, 8–15. [Google Scholar] [CrossRef]
- Xu, B.; Wang, W.; Falzon, G.; Kwan, P.; Guo, L.; Chen, G.; Tait, A.; Schneider, D. Automated cattle counting using Mask R-CNN in quadcopter vision system. Comput. Electron. Agric. 2020, 171. [Google Scholar] [CrossRef]
Date | Sensor | Ground Sampling Distance (m) | Bit Depth | Processing Level |
---|---|---|---|---|
10 March 2015 | WorldView-3 | 0.5 | 16 bits | Ortho ready standard product |
21 June 2015 | WorldView-3 | 0.5 | 16 bits | Ortho ready standard product |
3 June 2016 | WorldView-3 | 0.3 | 16 bits | Ortho ready standard product |
12 February 2017 | WorldView-2 | 0.5 | 16 bits | Ortho ready standard product |
June 2015 trained and tested on samples from the same image | |||||||||||||
Model weights | Pixel-based metrics | Object-based metrics | |||||||||||
F1 | MIoU | Ref count | Pred count | ||||||||||
Scratch | 0.845 | 0.809 | 0.804 | 0.493 | 10,383 | 9669 | 7075 | 2594 | 3296 | 0.682 | 0.732 | 0.706 | 0.546 |
ImageNet | 0.865 | 0.871 | 0.857 | 0.593 | 10,383 | 10,015 | 8843 | 1172 | 1513 | 0.854 | 0.883 | 0.868 | 0.767 |
COCO | 0.856 | 0.894 | 0.866 | 0.613 | 10,383 | 10,676 | 8914 | 1762 | 1484 | 0.857 | 0.835 | 0.846 | 0.733 |
Trained on June 2015 image and transferred to June 2016 image | |||||||||||||
Model weights | Pixel-based metrics | Object-based metrics | |||||||||||
F1 | MIoU | Ref count | Pred count | ||||||||||
Scratch | 0.792 | 0.850 | 0.793 | 0.476 | 17553 | 21601 | 13087 | 8514 | 3864 | 0.772 | 0.606 | 0.679 | 0.514 |
ImageNet | 0.851 | 0.886 | 0.857 | 0.595 | 17553 | 16158 | 14428 | 1730 | 2949 | 0.830 | 0.893 | 0.860 | 0.755 |
COCO | 0.863 | 0.901 | 0.872 | 0.629 | 17553 | 16655 | 14720 | 1935 | 2450 | 0.857 | 0.884 | 0.870 | 0.770 |
Trained on June 2015 image and transferred to February 2017 image | |||||||||||||
Model weights | Pixel-based metrics | Object-based metrics | |||||||||||
F1 | MIoU | Ref count | Pred count | ||||||||||
Scratch | 0.842 | 0.730 | 0.753 | 0.387 | 20912 | 16649 | 11818 | 4831 | 9088 | 0.565 | 0.710 | 0.629 | 0.459 |
ImageNet | 0.847 | 0.760 | 0.784 | 0.432 | 20912 | 17983 | 15403 | 2580 | 5895 | 0.723 | 0.857 | 0.784 | 0.645 |
COCO | 0.850 | 0.812 | 0.820 | 0.507 | 20912 | 18239 | 15743 | 2496 | 5309 | 0.748 | 0.863 | 0.801 | 0.669 |
Test March 2015 | MIoU | Ref count | Pred count | |||||||
0.669 | 19,002 | 14,077 | 13,902 | 175 | 4844 | 0.742 | 0.988 | 0.847 | 0.735 | |
Transfer | ||||||||||
June 2015 | 0.460 | 8146 | 6194 | 5988 | 206 | 2017 | 0.748 | 0.967 | 0.843 | 0.729 |
June 2016 | 0.546 | 17,553 | 13,826 | 12,728 | 1098 | 4579 | 0.735 | 0.921 | 0.818 | 0.692 |
February 2017 | 0.465 | 20,912 | 17,086 | 14,845 | 2241 | 5992 | 0.712 | 0.869 | 0.783 | 0.643 |
Test, June and March 2015 | MIoU | Ref count | Pred count | |||||||
0.625 | 28,656 | 24,782 | 22,693 | 2089 | 5664 | 0.800 | 0.916 | 0.854 | 0.745 | |
Transfer | ||||||||||
June 2016 | 0.626 | 17,553 | 15,928 | 14,247 | 1681 | 3059 | 0.823 | 0.895 | 0.857 | 0.750 |
February 2017 | 0.518 | 20,912 | 19,569 | 16,532 | 3037 | 4580 | 0.783 | 0.845 | 0.812 | 0.684 |
Test, March 2015 and June 2016 | MIoU | Ref count | Pred count | |||||||
0.466 | 8146 | 6658 | 6326 | 332 | 1614 | 0.797 | 0.950 | 0.867 | 0.765 | |
Transfer | ||||||||||
February 2017 | 0.532 | 20,912 | 17,104 | 15,440 | 1664 | 4572 | 0.772 | 0.903 | 0.832 | 0.712 |
MIoU | |||||||
Training (Source) | Transfer (Target) | Without Fine-Tune | With Fine-Tune | Without Fine-Tune | With Fine-Tune | ||
June 2015 | June 2016 | 0.629 | 0.638 | 0.93 | 0.770 | 0.762 | −0.77 |
June 2015 | February 2017 | 0.507 | 0.551 | 4.39 | 0.669 | 0.720 | 5.05 |
March 2015 | June 2015 | 0.460 | 0.508 | 4.83 | 0.729 | 0.734 | 0.52 |
March 2015 | February 2017 | 0.465 | 0.525 | 5.98 | 0.643 | 0.705 | 6.18 |
Studies | F1 | MIoU | Model | |||
[28] | - | - | - | - | 0.78 | Mask RCNN |
[72] | - | - | 0.578–0.881 * | - | - | Mask RCNN |
[72] | - | - | 0.568–0.879 * | - | - | Mask R-CNN with boundary regularization |
[38] | - | - | 0.390–0.930 * | - | - | Mask R-CNN with point rendering in mask head |
[71] | - | - | - | 0.48–0.70 * | - | Mask R-CNN |
[83] | 0.863 | 0.895 | 0.878 | - | - | Mask RCNN (ResNet101 backbone) |
[83] | 0.872 | 0.904 | 0.887 | - | - | Mask RCNN with rotation anchors and replacement of 2D convulation with receptive field block (RFB) in mask head |
[76] | - | - | 0.923 | - | 0.936 | Multiconstraint graph segmentation |
Ours (test) ** | 0.856 | 0.894 | 0.866 | 0.613 | 0.846 | Mask R-CNN (ResNet101 backbone) |
Ours (Transfer 1) ** | 0.863 | 0.901 | 0.872 | 0.629 | 0.870 | Mask R-CNN (ResNet101 backbone) |
Ours (Transfer 2) ** | 0.850 | 0.812 | 0.820 | 0.507 | 0.801 | Mask R-CNN (ResNet101 backbone) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gella, G.W.; Wendt, L.; Lang, S.; Tiede, D.; Hofer, B.; Gao, Y.; Braun, A. Mapping of Dwellings in IDP/Refugee Settlements from Very High-Resolution Satellite Imagery Using a Mask Region-Based Convolutional Neural Network. Remote Sens. 2022, 14, 689. https://doi.org/10.3390/rs14030689
Gella GW, Wendt L, Lang S, Tiede D, Hofer B, Gao Y, Braun A. Mapping of Dwellings in IDP/Refugee Settlements from Very High-Resolution Satellite Imagery Using a Mask Region-Based Convolutional Neural Network. Remote Sensing. 2022; 14(3):689. https://doi.org/10.3390/rs14030689
Chicago/Turabian StyleGella, Getachew Workineh, Lorenz Wendt, Stefan Lang, Dirk Tiede, Barbara Hofer, Yunya Gao, and Andreas Braun. 2022. "Mapping of Dwellings in IDP/Refugee Settlements from Very High-Resolution Satellite Imagery Using a Mask Region-Based Convolutional Neural Network" Remote Sensing 14, no. 3: 689. https://doi.org/10.3390/rs14030689