Dual-Task Semantic Change Detection for Remote Sensing Images Using the Generative Change Field Module
Abstract
:1. Introduction
2. Materials and Methods
2.1. Siamese Convolutional Network
2.2. General Networks for Dual-Task Semantic Change Detection
2.2.1. UNet-SCD
2.2.2. PSPNet-SCD
2.3. Generative Change Field Network for Dual-Task Semantic Change Detection
2.4. Dual-Task Semantic Change Detection Loss Function
2.4.1. WCE_Loss
2.4.2. Separable Loss
2.4.3. Union Loss
3. Results
3.1. Implementation Details
3.2. Dataset
3.3. Metrics
3.4. Effect of the GCF Module
3.5. Performance Analysis of Separable Loss
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Yang, J.; Weisberg, P.J.; Bristow, N.A. Landsat remote sensing approaches for monitoring long-term tree cover dynamics in semi-arid woodlands: Comparison of vegetation indices and spectral mixture analysis. Remote Sens. Environ. 2012, 119, 62–71. [Google Scholar] [CrossRef]
- Xian, G.; Homer, C. Updating the 2001 national land cover database impervious surface products to 2006 using landsat imagery change detection methods. Remote Sens. Environ. 2010, 114, 1676–1686. [Google Scholar] [CrossRef]
- Liang, B.; Weng, Q. Assessing urban environmental quality change of Indianapolis, United States, by the remote sensing and gis integration. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2011, 4, 43–55. [Google Scholar] [CrossRef]
- Wang, M.; Tan, K.; Jia, X.; Wang, X.; Chen, Y. A deep siamese network with hybrid convolutional feature extraction module for change detection based on multi-sensor remote sensing images. Remote Sens. 2020, 12, 205. [Google Scholar] [CrossRef] [Green Version]
- Chen, Q.; Chen, Y. Multi-feature object-based change detection using self-adaptive weight change vector analysis. Remote Sens. 2016, 8, 549. [Google Scholar] [CrossRef] [Green Version]
- Robin, A.; Moisan, L.; Le Hegarat-Mascle, S. An a-contrario approach for subpixel change detection in satellite imagery. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 1977–1993. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lanza, A.; Di Stefano, L. Statistical change detection by the pool adjacent violators algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33, 1894–1910. [Google Scholar] [CrossRef] [PubMed]
- Lingg, A.J.; Zelnio, E.; Garber, F.; Rigling, B.D. A sequential framework for image change detection. IEEE Trans. Image Process. 2014, 23, 2405–2413. [Google Scholar] [CrossRef]
- Prendes, J.; Chabert, M.; Pascal, F.; Giros, A.; Tourneret, J.-Y. A new multivariate statistical model for change detection in images acquired by homogeneous and heterogeneous sensors. IEEE Trans. Image Process. 2015, 24, 799–812. [Google Scholar] [CrossRef] [Green Version]
- Liu, S.-C.; Fu, C.-W.; Chang, S. Statistical change detection with moments under time-varying illumination. IEEE Trans. Image Process. 1998, 7, 1258–1268. [Google Scholar] [CrossRef]
- Chatelain, F.; Tourneret, J.-Y.; Inglada, J.; Ferrari, A. Bivariate gamma distributions for image registration and change detection. IEEE Trans. Image Process. 2007, 16, 1796–1806. [Google Scholar] [CrossRef] [Green Version]
- Huang, X.; Zhang, L.; Zhu, T. Building change detection from multitemporal high-resolution remotely sensed images based on a morphological building index. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 105–115. [Google Scholar] [CrossRef]
- Zhan, Y.; Fu, K.; Yan, M.; Sun, X.; Wang, H.; Qiu, X. Change detection based on deep siamese convolutional network for optical aerial images. IEEE Geosci. Remote Sens. Lett. 2017, 14, 1845–1849. [Google Scholar] [CrossRef]
- Lv, Z.Y.; Shi, W.; Zhang, X.; Benediktsson, J.A. Landslide inventory mapping from bitemporal high-resolution remote sensing images using change detection and multiscale segmentation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 1520–1532. [Google Scholar] [CrossRef]
- Daudt, R.C.; Le Saux, B.; Boulch, A. Fully convolutional siamese networks for change detection. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–8 October 2018; pp. 4063–4067. [Google Scholar]
- Liu, Y.; Pang, C.; Zhan, Z.; Zhang, X.; Yang, X. Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model. IEEE Geosci. Remote Sens. Lett. 2021, 18, 811–815. [Google Scholar] [CrossRef]
- Chen, J.; Yuan, Z.; Peng, J.; Chen, L.; Huang, H.; Zhu, J.; Liu, Y.; Li, H. DASNet: Dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 1194–1206. [Google Scholar] [CrossRef]
- Daudt, R.C.; Le Saux, B.; Boulch, A.; Gousseau, Y. Multitask learning for large-scale semantic change detection. Comput. Vis. Image Underst. 2019, 187, 102783. [Google Scholar] [CrossRef] [Green Version]
- Mou, L.; Bruzzone, L.; Zhu, X.X. Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery. IEEE Trans. Geosci. Remote Sens. 2019, 57, 924–935. [Google Scholar] [CrossRef] [Green Version]
- Yang, K.; Xia, G.-S.; Liu, Z.; Du, B.; Yang, W.; Pelillo, M. Asymmetric Siamese Networks for Semantic Change Detection. arXiv 2020, arXiv:2010.05687. [Google Scholar]
- Yao, J.; Cao, X.; Zhao, Q.; Meng, D.; Xu, Z. Robust subspace clustering via penalized mixture of Gaussians. Neurocomputing 2018, 278, 4–11. [Google Scholar] [CrossRef]
- Hong, D.; Gao, L.; Yao, J.; Zhang, B.; Plaza, A.; Chanussot, J. Graph convolutional networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2021, 59, 5966–5978. [Google Scholar] [CrossRef]
- Sun, Y.; Zhang, X.; Huang, J.; Wang, H.; Xin, Q. Fine-grained building change detection from very high-spatial-resolution remote sensing images based on deep multitask learning. IEEE Geosci. Remote Sens. Lett. 2020, 1–5. [Google Scholar] [CrossRef]
- Peng, D.; Bruzzone, L.; Zhang, Y.; Guan, H.; Ding, H.; Huang, X. SemiCDNet: A semisupervised convolutional neural network for change detection in high resolution remote-sensing images. IEEE Trans. Geosci. Remote Sens. 2021, 59, 5891–5906. [Google Scholar] [CrossRef]
- Liu, J.; Gong, M.; Qin, K.; Zhang, P. A deep convolutional coupling network for change detection based on heterogeneous optical and radar images. IEEE Trans. Neural Netw. Learn. Syst. 2018, 29, 545–559. [Google Scholar] [CrossRef] [PubMed]
- Chen, H.; Wu, C.; Du, B.; Zhang, L.; Wang, L. Change detection in multisource VHR images via deep siamese convolutional multiple-layers recurrent neural network. IEEE Trans. Geosci. Remote Sens. 2020, 58, 2848–2864. [Google Scholar] [CrossRef]
- Graves, A. Generating sequences with recurrent neural networks. arXiv 2014, arXiv:1308.0850. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain, 16–20 September 2015; Springer: Berlin/Heidelberg, Germany, 2015; pp. 234–241. [Google Scholar] [CrossRef] [Green Version]
- Zhao, H.; Shi, J.; Qi, X.; Wang, X.; Jia, J. Pyramid scene parsing network. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 630–645. [Google Scholar]
- Robbins, H.; Monro, S. A stochastic approximation method. In Herbert Robbins Selected Papers; Springer: New York, NY, USA, 1985; Volume 1, pp. 102–109. [Google Scholar] [CrossRef]
- Lopez-Fandino, J.; Garea, A.S.; Heras, D.B.; Arguello, F. Stacked autoencoders for multiclass change detection in hyperspectral images. In Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 1906–1909. [Google Scholar]
- Saha, S.; Bovolo, F.; Brurzone, L. Unsupervised multiple-change detection in VHR optical images using deep features. In Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 1902–1905. [Google Scholar]
- Benedek, C.; Sziranyi, T. Change detection in optical aerial images by a multilayer conditional mixed markov model. IEEE Trans. Geosci. Remote Sens. 2009, 47, 3416–3430. [Google Scholar] [CrossRef] [Green Version]
- Chen, H.; Shi, Z. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sens. 2020, 12, 1662. [Google Scholar] [CrossRef]
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2999–3007. [Google Scholar]
- Solberg, A.; Taxt, T.; Jain, A. A markov random field model for classification of multisource satellite imagery. IEEE Trans. Geosci. Remote Sens. 1996, 34, 100–113. [Google Scholar] [CrossRef]
- Li, A.; Jiao, L.; Zhu, H.; Li, L.; Liu, F. Multitask semantic boundary awareness network for remote sensing image segmentation. IEEE Trans. Geosci. Remote Sens. 2021, 1–14. [Google Scholar] [CrossRef]
Number | Abbreviation | Explanation |
---|---|---|
1 | R | Set of spatial domains |
2 | C | Channel |
3 | H | Height |
4 | W | Width |
5 | Learning parameters of network | |
6 | Concat/⊕ | Feature map concatenation |
7 | Conv | Convolutional layer |
8 | ReLU | Rectified Linear Units |
9 | BN | Batch normalization |
10 | p | Probability of prediction |
11 | y | Ground truth |
Name | Types | Filters | Output Size |
---|---|---|---|
Layer0 | Conv + BN + ReLU + Maxpool | 64 | 64 × 64 × 64 |
Layer1 | Residual block × 3 | 64 | 64 × 64 × 64 |
Layer2 | Residual block × 4 | 128 | 128 × 32 × 32 |
Name | Types | Output Filters | Output Size |
---|---|---|---|
Concat | Conv + BN + ReLU + Conv | 128 × 3, 128 | 128 × 32 × 32 |
Layer3 | Residual block × 6 | 256 | 256× 32 × 32 |
Layer4 | Residual block × 3 | 512 | 512 × 32 × 32 |
PPM | Adaptivepool × 4 | 2560 | 2560 × 32 × 32 |
Conv | Conv + BN + ReLU | 512 | 512 × 32 × 32 |
Output3 | Conv + BN + ReLU + Conv | 512, 2 | 2 × 32 × 32 |
Upsample | Bilinear interpolation | _ | 2 × 256 × 256 |
Seg1 | Conv + BN + ReLU + Conv | 512, 512 | 512 × 32 × 32 |
Output1 | Conv + BN + ReLU + Conv | 512, 7 | 7 × 32× 32 |
Upsample | Bilinear interpolation | _ | 7 × 256 × 256 |
Seg2 | Conv + BN + ReLU + Conv | 512, 512 | 512 × 32 × 32 |
Output2 | Conv + BN + ReLU + Conv | 512, 7 | 7 × 32 × 32 |
Upsample | Bilinear interpolation | _ | 7 × 256 × 256 |
Methods | OA | IoU1 | IoU2 | mIoU | SeK | Flops | Parameter |
---|---|---|---|---|---|---|---|
FC-EF [15] | 83.7 | 84.2 | 43.0 | 63.6 | 8.7 | 62.9G | 17.59M |
FC-Siam-conc [15] | 84.6 | 85.0 | 45.7 | 65.3 | 11.4 | 62.9G | 17.59M |
FC-Siam-diff [15] | 84.5 | 85.2 | 46.7 | 65.9 | 11.4 | 24.5G | 17.59M |
UNet-SCD | 83.3 | 83.5 | 42.5 | 63.0 | 9.2 | 19.6G | 21.87M |
PSPNet-SCD | 85.0 | 85.4 | 47.9 | 66.7 | 13.2 | 56.3G | 25.55M |
GCF-SCD-Net | 85.3 | 85.9 | 49.3 | 67.6 | 14.1 | 56.1G | 25.57M |
Methods | WCE_Loss | Focal Loss | Separable Loss | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
OA | IoU1 | IoU2 | mIoU | SeK | OA | IoU1 | IoU2 | mIoU | SeK | OA | IoU1 | IoU2 | mIoU | SeK | |
FC-EF [15] | 83.0 | 83.7 | 44.7 | 64.2 | 8.9 | 82.6 | 83.2 | 43.2 | 63.2 | 7.9 | 82.9 | 83.5 | 46.3 | 64.9 | 9.8 |
FC-Siam-conc [15] | 84.1 | 84.6 | 48.2 | 66.4 | 12.6 | 83.3 | 83.7 | 46.5 | 65.1 | 11.1 | 84.3 | 84.8 | 49 | 66.9 | 13.2 |
FC-Siam-diff [15] | 84.2 | 84.8 | 48.6 | 66.7 | 12.7 | 83.5 | 84.1 | 46.7 | 65.4 | 11.0 | 84.3 | 84.9 | 49.5 | 67.2 | 13.4 |
UNet-SCD | 83.4 | 83.5 | 43.2 | 63.4 | 9.8 | 82.7 | 82.8 | 42.2 | 62.5 | 8.8 | 83.3 | 83.4 | 44.1 | 63.8 | 10.2 |
PSPNet-SCD | 84.8 | 85.3 | 50.1 | 67.7 | 14.5 | 84.2 | 84.7 | 48.5 | 66.6 | 12.9 | 84.9 | 85.4 | 52.0 | 68.7 | 15.9 |
GCF-SCD-Net | 85.2 | 85.8 | 50.7 | 68.3 | 15.0 | 84.3 | 84.8 | 49.7 | 67.3 | 13.9 | 85.3 | 85.8 | 52.4 | 69.1 | 16.5 |
Methods | Flip 🗴 | Flip ✓ | ||
---|---|---|---|---|
mIoU | SeK | mIoU | SeK | |
FC-EF | 64.9 | 9.8 | 65.4 | 10.5 |
FC-Siam-conc | 66.9 | 13.2 | 67.4 | 14.1 |
FC-Siam-diff | 67.2 | 13.4 | 67.7 | 14.2 |
HRSCD.str1 (reported by [20]) | 29.3 | 4.6 | 29.8 | 4.9 |
HRSCD.str2 (reported by [20]) | 59.7 | 6.3 | 59.4 | 6.6 |
HRSCD.str3 (reported by [20]) | 62.3 | 8.9 | 62.1 | 9.2 |
HRSCD.str4 (reported by [20]) | 67.5 | 13.7 | 67.9 | 14.5 |
ASN [20] | 69.0 | 15.2 | 69.7 | 16.2 |
ASN-ATL [20] | 69.1 | 15.5 | 70.0 * | 16.8 |
GCF-SCD-Net | 69.1 | 16.5 | 69.9 | 17.9 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xiang, S.; Wang, M.; Jiang, X.; Xie, G.; Zhang, Z.; Tang, P. Dual-Task Semantic Change Detection for Remote Sensing Images Using the Generative Change Field Module. Remote Sens. 2021, 13, 3336. https://doi.org/10.3390/rs13163336
Xiang S, Wang M, Jiang X, Xie G, Zhang Z, Tang P. Dual-Task Semantic Change Detection for Remote Sensing Images Using the Generative Change Field Module. Remote Sensing. 2021; 13(16):3336. https://doi.org/10.3390/rs13163336
Chicago/Turabian StyleXiang, Shao, Mi Wang, Xiaofan Jiang, Guangqi Xie, Zhiqi Zhang, and Peng Tang. 2021. "Dual-Task Semantic Change Detection for Remote Sensing Images Using the Generative Change Field Module" Remote Sensing 13, no. 16: 3336. https://doi.org/10.3390/rs13163336
APA StyleXiang, S., Wang, M., Jiang, X., Xie, G., Zhang, Z., & Tang, P. (2021). Dual-Task Semantic Change Detection for Remote Sensing Images Using the Generative Change Field Module. Remote Sensing, 13(16), 3336. https://doi.org/10.3390/rs13163336