TriangleConv: A Deep Point Convolutional Network for Recognizing Building Shapes in Map Space
Abstract
:1. Introduction
2. The Deep Point Convolutional Network
2.1. The Framework of DPCN
- Point feature extraction module: this module aims to learn and extract the feature representations of each point of the input building. As shown in Figure 1, it stacks two TriangleConv layers. Each TriangleConv layer performs the TriangleConv operator on the points. The first TriangleConv layer generates a feature representation with 64 dimensions for each point. Taking this new feature representation as input, the second TriangleConv layer extracts a higher dimensional feature representation with 1024 dimensions for each point. Through these TriangleConv layers, DPCN embeds each point of the input building into a high dimensional space.
- Building feature extraction module: this module is to aggregate the features of the points and obtain the feature representation of the input building. To achieve this purpose, a max pooling layer is used, which executes the max operation on each dimension of the feature representations of the points. Accordingly, a tensor with 1024 dimensions is derived to represent the deep features of the shape of the building. This feature representation will be used by the building shape recognition module to predict the shape of the input building.
- Building shape recognition module: this module aims to predict the shape of the input building with its feature representation. It usually stacks several fully connected layers and one softmax layer. As shown in Figure 1, there are three fully connected layers in DPCN. The fully connected layers transform a feature of one space into a new feature of another space and aggregate the information of different dimensions in this process. With three fully connected layers, the building shape feature representation with 1024 dimensions is transformed into a k dimensional representation where k is the number of shape classes. Finally, the softmax layer executes the softmax operation on the k dimensional representation to output the possibility scores that the input object belongs to different shape classes.
2.2. The TriangleConv Operator
2.3. The Implementation and the Parameters of Training
3. Dataset and Metrics
3.1. Experimental Dataset and Preprocessing
3.2. The Evaluation Metrics
4. Results and Analysis
4.1. The Sensitivity Analysis of the Number of Input Points of Each Building
4.2. The Performance Analysis of the Candidate Convolution Methods
4.3. Comparison with Related Methods
4.4. The Application of DPCN
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Du, S.; Zhang, F.; Zhang, X. Semantic classification of urban buildings combining VHR image and GIS data: An improved random forest approach. ISPRS J. Photogramm. Remote Sens. 2015, 105, 107–119. [Google Scholar] [CrossRef]
- Zhou, X.; Chen, Z.; Zhang, X.; Ai, T. Change Detection for Building Footprints with Different Levels of Detail Using Combined Shape and Pattern Analysis. ISPRS Int. J. Geo-Inf. 2018, 7, 406. [Google Scholar] [CrossRef] [Green Version]
- Shea, K.S.; Mcmaster, R.B. Cartographic Generalization in a Digital Environment: When and How to Generalize. J. Fluid Mech. 1989, 1, 56–67. [Google Scholar]
- Mark, D.M.; Freksa, C.; Hirtle, S.C.; Lloyd, R.; Tversky, B. Cognitive models of geographical space. Int. J. Geogr. Inf. Sci. 1999, 13, 747–774. [Google Scholar] [CrossRef]
- Matikainen, L.; Hyyppä, J.; Ahokas, E.; Markelin, L.; Kaartinen, H. Automatic Detection of Buildings and Changes in Buildings for Updating of Maps. Remote Sens. 2010, 2, 1217–1248. [Google Scholar] [CrossRef] [Green Version]
- Touya, G.; Zhang, X.; Lokhat, I. Is deep learning the new agent for map generalization? Int. J. Cartogr. 2019, 5, 142–157. [Google Scholar] [CrossRef] [Green Version]
- Rainsford, D.; Mackaness, W. Template Matching in Support of Generalisation of Rural Buildings. In Advances in Spatial Data Handling; Richardson, D.E., van Oosterom, P., Eds.; Springer: Berlin/Heidelberg, Germany, 2002; pp. 137–151. [Google Scholar]
- Wang, H.L.; Wu, F.; Zhang, L.L.; Deng, H.Y. The application of mathematical morphology and pattern recognition to building polygon simplification. Acta Geod. Cartogr. Sin. 2005, 34, 269–276. [Google Scholar]
- Basaraner, M.; Cetinkaya, S. Performance of shape indices and classification schemes for characterising perceptual shape complexity of building footprints in GIS. Int. J. Geogr. Inf. Sci. 2017, 31, 1952–1977. [Google Scholar] [CrossRef]
- Yang, C.; Wei, H.; Yu, Q. A novel method for 2D nonrigid partial shape matching. Neurocomputing 2018, 275, 1160–1176. [Google Scholar] [CrossRef]
- Niu, X.X.; Suen, C.Y. A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recognit. 2012, 45, 1318–1325. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2015, arXiv:1409.1556. [Google Scholar]
- Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 2015, 115, 211–252. [Google Scholar] [CrossRef] [Green Version]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Kalchbrenner, N.; Grefenstette, E.; Blunsom, P. A Convolutional Neural Network for Modelling Sentences. arXiv 2014, arXiv:1404.2188. [Google Scholar]
- Kim, Y.; Jernite, Y.; Sontag, D.; Rush, A.M. Character-Aware neural language models. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, AAAI 2016, Phoenix, AZ, USA, 12–17 February 2016; AAAI Press: San Francisco, CA, USA, 2016; pp. 2741–2749. [Google Scholar]
- LeCun, Y.; Bengio, Y. Convolutional networks for images, speech, and time series. Handb. Brain Theory Neural Netw. 1995, 3361, 1995. [Google Scholar]
- Li, X.; Zheng, H.; Han, C.; Zheng, W.; Chen, H.; Jing, Y.; Dong, K. SFRS-Net: A Cloud-Detection Method Based on Deep Convolutional Neural Networks for GF-1 Remote-Sensing Images. Remote Sens. 2021, 13, 2910. [Google Scholar] [CrossRef]
- Lazer, D.; Pentland, A.S.; Adamic, L.; Aral, S.; Barabasi, A.L.; Brewer, D.; Christakis, N.; Contractor, N.; Fowler, J.; Gutmann, M. Life in the network: The coming age of computational social science. Science 2009, 323, 721–723. [Google Scholar] [CrossRef] [Green Version]
- Xu, B.; Cen, K.; Huang, J.; Shen, H.; Cheng, X. A survey on graph convolutional neural network. Chin. J. Comput. 2020, 43, 755–780. [Google Scholar]
- Bruna, J.; Zaremba, W.; Szlam, A.; LeCun, Y. Spectral Networks and Locally Connected Networks on Graphs. arXiv 2014, arXiv:1312.6203. [Google Scholar]
- Abdel-Hamid, O.; Mohamed, A.R.; Jiang, H.; Deng, L.; Penn, G.; Yu, D. Convolutional Neural Networks for Speech Recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 2014, 22, 1533–1545. [Google Scholar] [CrossRef] [Green Version]
- Kipf, T.N.; Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. arXiv 2017, arXiv:1609.02907. [Google Scholar]
- Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Liò, P.; Bengio, Y. Graph Attention Networks. arXiv 2018, arXiv:1710.10903. [Google Scholar]
- Hamilton, W.L.; Ying, R.; Leskovec, J. Inductive Representation Learning on Large Graphs. arXiv 2018, arXiv:1706.02216. [Google Scholar]
- Zhang, C.; Wang, J.; Yao, K. Global Random Graph Convolution Network for Hyperspectral Image Classification. Remote Sens. 2021, 13, 2285. [Google Scholar] [CrossRef]
- Yan, X.; Ai, T.; Yang, M.; Yin, H. A graph convolutional neural network for classification of building patterns using spatial vector data. ISPRS J. Photogramm. Remote Sens. 2019, 150, 259–273. [Google Scholar] [CrossRef]
- Yan, X.; Ai, T.; Yang, M.; Tong, X. Graph convolutional autoencoder model for the shape coding and cognition of buildings in maps. Int. J. Geogr. Inf. Sci. 2021, 35, 490–512. [Google Scholar] [CrossRef]
- Qi, C.R.; Su, H.; Mo, K.; Guibas, L.J. PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 652–660. [Google Scholar]
- Qi, C.R.; Yi, L.; Su, H.; Guibas, L.J. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. arXiv 2017, arXiv:1706.02413. [Google Scholar]
- Wang, Y.; Sun, Y.; Liu, Z.; Sarma, S.E.; Bronstein, M.M.; Solomon, J.M. Dynamic Graph CNN for Learning on Point Clouds. ACM Trans. Graph. 2019, 38. [Google Scholar] [CrossRef] [Green Version]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv 2019, arXiv:1912.01703. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2017, arXiv:1412.6980. [Google Scholar]
- Yan, X.; Ai, T.; Zhang, X. Template Matching and Simplification Method for Building Features Based on Shape Cognition. ISPRS Int. J. Geo-Inf. 2017, 6, 250. [Google Scholar] [CrossRef] [Green Version]
- Douglas, D.H.; Peucker, T.K. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartogr. Int. J. Geogr. Inf. Geovis. 1973, 10, 112–122. [Google Scholar] [CrossRef] [Green Version]
- DGL Development Team. Deep Graph Library. 2018. Available online: https://docs.dgl.ai/en/0.5.x/index.html (accessed on 3 February 2021).
Point Number | Accuracy | Macro-Recall | Macro-Precision | Macro-F1 |
---|---|---|---|---|
2 | 0.3941 | 0.6731 | 0.3689 | 0.3273 |
4 | 0.8624 | 0.8537 | 0.5437 | 0.5955 |
8 | 0.9624 | 0.9872 | 0.7486 | 0.8137 |
16 | 0.9842 | 0.9902 | 0.8376 | 0.8874 |
32 | 0.9752 | 0.9882 | 0.8079 | 0.8664 |
64 | 0.9762 | 0.9881 | 0.7914 | 0.8509 |
128 | 0.9733 | 0.9871 | 0.7300 | 0.8077 |
Accuracy | Macro-Recall | Macro-Precision | Macro-F1 | |
---|---|---|---|---|
DPCN-3point | 0.9653 | 0.9858 | 0.7443 | 0.817 |
DPCN | 0.9842 | 0.9902 | 0.8376 | 0.8874 |
Accuracy | Macro-Recall | Macro-Precision | Macro-F1 | |
---|---|---|---|---|
GCN+F | 0.9505 | 0.9827 | 0.6877 | 0.7689 |
GAT+F | 0.9505 | 0.9886 | 0.7455 | 0.8041 |
GraphSAGE+F | 0.9574 | 0.9902 | 0.7498 | 0.8173 |
GCN | 0.3396 | 0.3224 | 0.2835 | 0.1802 |
GAT | 0.6465 | 0.4544 | 0.3298 | 0.3015 |
GraphSAGE | 0.5010 | 0.7598 | 0.5116 | 0.4780 |
PointNet | 0.9614 | 0.9243 | 0.7622 | 0.8048 |
PointNet++ | 0.9505 | 0.9637 | 0.6999 | 0.7623 |
DGCNN | 0.9634 | 0.9348 | 0.7135 | 0.7790 |
DPCN | 0.9842 | 0.9902 | 0.8376 | 0.8874 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, C.; Hu, Y.; Li, Z.; Xu, J.; Han, Z.; Guo, J. TriangleConv: A Deep Point Convolutional Network for Recognizing Building Shapes in Map Space. ISPRS Int. J. Geo-Inf. 2021, 10, 687. https://doi.org/10.3390/ijgi10100687
Liu C, Hu Y, Li Z, Xu J, Han Z, Guo J. TriangleConv: A Deep Point Convolutional Network for Recognizing Building Shapes in Map Space. ISPRS International Journal of Geo-Information. 2021; 10(10):687. https://doi.org/10.3390/ijgi10100687
Chicago/Turabian StyleLiu, Chun, Yaohui Hu, Zheng Li, Junkui Xu, Zhigang Han, and Jianzhong Guo. 2021. "TriangleConv: A Deep Point Convolutional Network for Recognizing Building Shapes in Map Space" ISPRS International Journal of Geo-Information 10, no. 10: 687. https://doi.org/10.3390/ijgi10100687
APA StyleLiu, C., Hu, Y., Li, Z., Xu, J., Han, Z., & Guo, J. (2021). TriangleConv: A Deep Point Convolutional Network for Recognizing Building Shapes in Map Space. ISPRS International Journal of Geo-Information, 10(10), 687. https://doi.org/10.3390/ijgi10100687