# Learning Cartographic Building Generalization with Deep Convolutional Neural Networks

^{*}

## Abstract

**:**

## 1. Introduction

## 2. State of the Art

## 3. Approach

#### 3.1. U-net

#### 3.2. Residual U-net

#### 3.3. Generative Adversarial Network (GAN)

## 4. Experiments

#### 4.1. Data Preparation

#### 4.2. Training the Network

#### 4.2.1. Loss Function

#### 4.2.2. Optimizer and Learning Rate

#### 4.2.3. GAN: Discriminator and Weighting Factor

## 5. Results and Discussion

#### 5.1. Quantitative Analyses on a Single Independent Test Area at Three Target Map Scales

#### 5.2. Experiments with the Specific Map Scale 1:15,000

#### 5.3. Experiments with Specific Buildings and Orientations

#### 5.4. Map Series in Different Scales

#### 5.5. Results with a Large Extent of the Data Set

#### 5.6. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

CNN | convolutional neural network |

DCNN | deep convolutional neural network |

GAN | generative adversarial network |

GPU | graphics processing unit |

LSTM | long short-term memory |

MAE | mean absolute error |

MSE | mean squared error |

OSM | OpenStreetMap |

ResNet | deep residual network |

## References

- Badrinarayanan, V.; Kendall, A.; Cipolla, R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv
**2015**, arXiv:1511.00561. [Google Scholar] [CrossRef] [PubMed] - Mackaness, W.A.; Ruas, A.; Sarjakoski, L.T. Generalisation of Geographic Information: Cartographic Modelling and Applications; Elsevier: Amsterdam, The Netherlands, 2011. [Google Scholar]
- Sester, M. Optimization approaches for generalization and data abstraction. Int. J. Geogr. Inf. Sci.
**2005**, 19, 871–897. [Google Scholar] [CrossRef] - Stoter, J.; Post, M.; van Altena, V.; Nijhuis, R.; Bruns, B. Fully automated generalization of a 1:50 k map from 1:10 k data. Cartogr. Geogr. Inf. Sci.
**2014**, 41, 1–13. [Google Scholar] [CrossRef] - Renard, J.; Gaffuri, J.; Duchêne, C.; Touya, G. Automated generalisation results using the agent-based platform CartAGen. In Proceedings of the 25th International Cartographic Conference (ICC’11), Paris, France, 3–8 July 2011. [Google Scholar]
- Müller, J.C.; Zeshen, W. Area-patch generalisation: A competitive approach. Cartogr. J.
**1992**, 29, 137–144. [Google Scholar] [CrossRef] - Jäger, E. Investigations on automated feature displacement for small scale maps in raster format. In Proceedings of the 15th Conference of the ICA, Bournemouth, UK, 22 September–1 October 1991; pp. 245–256. [Google Scholar]
- Damen, J.; van Kreveld, M.; Spaan, B. High quality building generalization by extending the morphological operators. In Proceedings of the 11th ICA Workshop on Generalization and Multiple Representation, Montpellier, France, 20–21 June 2008; pp. 1–12. [Google Scholar]
- Li, Z.; Yan, H.; Ai, T.; Chen, J. Automated building generalization based on urban morphology and Gestalt theory. Int. J. Geogr. Inf. Sci.
**2004**, 18, 513–534. [Google Scholar] [CrossRef] - Shen, Y.; Ai, T.; Li, W.; Yang, M.; Feng, Y. A polygon aggregation method with global feature preservation using superpixel segmentation. Comput. Environ. Urban Syst.
**2019**, 75, 117–131. [Google Scholar] [CrossRef] - Powitz, B. Computer-assisted generalization—An important software tool for gis. Int. Arch. Photogramm. Remote Sens.
**1993**, 29, 664–673. [Google Scholar] - Weibel, R.; Keller, S.F.; Reichenbacher, T. Overcoming the knowledge acquisition bottleneck in map generalization: The role of interactive systems and computational intelligence. In COSIT; Springer: Berlin/Heidelberg, Germany, 1995; pp. 139–156. [Google Scholar]
- Mustière, S. GALBE: Adaptive Generalisation. The need for an Adaptive Process for Automated Generalisation, an Example on Roads. In Proceedings of the Geographic Information Systems PlaNet, Lisbon, Portugal, 7–11 September 1998. [Google Scholar]
- Sester, M. Knowledge acquisition for the automatic interpretation of spatial data. Int. J. Geogr. Inf. Sci.
**2000**, 114, 1–24. [Google Scholar] [CrossRef] - Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE
**1998**, 86, 2278–2324. [Google Scholar] [CrossRef] [Green Version] - Zhong, Z.; Jin, L.; Xie, Z. High performance offline handwritten chinese character recognition using googlenet and directional feature maps. In Proceedings of the International Conference on Document Analysis and Recognition (ICDAR 2015), Nancy, France, 23–26 August 2015; pp. 846–850. [Google Scholar]
- Feng, Y.; Sester, M. Extraction of pluvial flood relevant volunteered geographic information (VGI) by deep learning from user generated texts and photos. ISPRS Int. J. Geo-Inf.
**2018**, 7, 39. [Google Scholar] [CrossRef] - Feng, Y.; Shebotnov, S.; Brenner, C.; Sester, M. Ensembled convolutional neural network models for retrieving flood relevant tweets. In Proceedings of the MediaEval 2018 Workshop, Sophia Antipolis, France, 29–31 October 2018; p. 27. [Google Scholar]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar]
- Marmanis, D.; Schindler, K.; Wegner, J.D.; Galliani, S.; Datcu, M.; Stilla, U. Classification with an edge: Improving semantic image segmentation with boundary detection. ISPRS J. Photogramm. Remote Sens.
**2018**, 135, 158–172. [Google Scholar] [CrossRef] [Green Version] - Chen, K.; Fu, K.; Yan, M.; Gao, X.; Sun, X.; Wei, X. Semantic Segmentation of Aerial Images with Shuffling Convolutional Neural Networks. IEEE Geosci. Remote Sens. Lett.
**2018**, 15, 173–177. [Google Scholar] [CrossRef] - Wang, W.; Yang, N.; Zhang, Y.; Wang, F.; Cao, T.; Eklund, P. A review of road extraction from remote sensing images. J. Traffic Transp. Eng.
**2016**, 3, 271–282. [Google Scholar] [CrossRef] [Green Version] - Xu, Y.; Chen, Z.; Xie, Z.; Wu, L. Quality assessment of building footprint data using a deep autoencoder network. Int. J. Geogr. Inf. Sci.
**2017**, 31, 1929–1951. [Google Scholar] [CrossRef] - Zhou, Q.; Li, Z. A comparative study of various supervised learning approaches to selective omission in a road network. Cartogr. J.
**2017**, 54, 254–264. [Google Scholar] [CrossRef] - Lee, J.; Jang, H.; Yang, J.; Yu, K. Machine Learning Classification of Buildings for Map Generalization. ISPRS Int. J. Geo-Inf.
**2017**, 6, 309. [Google Scholar] [CrossRef] - Cheng, B.; Liu, Q.; Li, X.; Wang, Y. Building simplification using backpropagation neural networks: A combination of cartographers’ expertise and raster-based local perception. Gisci. Remote Sens.
**2013**, 50, 527–542. [Google Scholar] [CrossRef] - Thiemann, F.; Cakir, S.; Politz, F. Generalisierung mittels Deep Learning (CNN) am Beispiel der Straßenextraktion aus GPS-Trajektorien. In 38. Wissenschaftlich-Technische Jahrestagung der DGPF und PFGK18 Tagung, Muenchen Publikationen der DGPF, Band 27; Deutsche Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation (DGPF) e.V.: Munich, Germany, 2018. [Google Scholar]
- Regnauld, N.; McMaster, R. A Synoptic View of Generalisation Operators. In Generalisation of Geographic Information; Mackaness, W., Ruas, A., Sarjakoski, L., Eds.; Elsevier Science B.V.: Amsterdam, The Netherlands, 2007; pp. 37–66. [Google Scholar]
- Sester, M.; Feng, Y.; Thiemann, F. Building generalization using deep learning. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2018**, XLII-4, 565–572. [Google Scholar] [CrossRef] - Simo-Serra, E.; Iizuka, S.; Sasaki, K.; Ishikawa, H. Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup. ACM Trans. Graph.
**2016**, 35, 121. [Google Scholar] [CrossRef] - Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Zhang, Z.; Liu, Q.; Wang, Y. Road Extraction by Deep Residual UNet. IEEE Geosci. Remote Sens. Lett.
**2018**, 15, 749–753. [Google Scholar] [CrossRef] - Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27; Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2014; pp. 2672–2680. [Google Scholar]
- Politz, F.; Sester, M. Exploring ALS and DIM data for semantic segmentation using CNNs. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2018**, 42, 347–354. [Google Scholar] [CrossRef] - Isola, P.; Zhu, J.Y.; Zhou, T.; Efros, A.A. Image-To-Image Translation With Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 5967–5976. [Google Scholar]
- Springenberg, J.T.; Dosovitskiy, A.; Brox, T.; Riedmiller, M. Striving for simplicity: The all convolutional net. In Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Identity mappings in deep residual networks. In Proceedings of the 14th European Conference on Computer Vision, Cham, The Netherlands, 8–16 October 2016; Springer: Berlin, Germany, 2016; pp. 630–645. [Google Scholar]
- Chen, Y.; Lai, Y.K.; Liu, Y.J. Cartoongan: Generative adversarial networks for photo cartoonization. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 9465–9474. [Google Scholar]
- Zhu, J.Y.; Park, T.; Isola, P.; Efros, A.A. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2242–2251. [Google Scholar]
- Peters, T.; Brenner, C. Conditional adversarial networks for multimodal photo-realistic point cloud rendering. In Proceedings of the Spatial Big Data and Machine Learning in GIScience, Workshop at GIScience 2018, Melbourne, Australia, 28 August 2018; pp. 48–53. [Google Scholar]
- Ledig, C.; Theis, L.; Huszár, F.; Caballero, J.; Cunningham, A.; Acosta, A.; Aitken, A.; Tejani, A.; Totz, J.; Wang, Z.; et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4681–4690. [Google Scholar]
- Simo-Serra, E.; Iizuka, S.; Ishikawa, H. Mastering sketching: Adversarial augmentation for structured prediction. ACM Trans. Graph. (TOG)
**2018**, 37, 11. [Google Scholar] [CrossRef] - Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv
**2014**, arXiv:1411.1784. [Google Scholar] - Arjovsky, M.; Chintala, S.; Bottou, L. Wasserstein gan. arXiv
**2017**, arXiv:1701.07875. [Google Scholar] - Kim, T.; Cha, M.; Kim, H.; Lee, J.K.; Kim, J. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; Volume 70, pp. 1857–1865. [Google Scholar]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
- Hinton, G.; Srivastava, N.; Swersky, K. Lecture 6a Overview of Mini-Batch Gradient Descent. Coursera Course Lectures: Neural Networks for Machine Learning. 2012. Available online: http://www.cs.toronto.edu/~hinton/coursera/lecture6/lec6.pdf (accessed on 30 May 2019).
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Zeiler, M.D. ADADELTA: An adaptive learning rate method. arXiv
**2012**, arXiv:1212.5701. [Google Scholar] - Sester, M.; Neidhart, H. Reconstruction of building ground plans from laser scanner data. In Proceedings of the 11th AGILE International Conference on Geographic Information Science 2008, Girona, Spain, 4–8 August 2008. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed] - Yan, X.; Ai, T.; Yang, M.; Yin, H. A graph convolutional neural network for classification of building patterns using spatial vector data. ISPRS J. Photogramm. Remote Sens.
**2019**, 150, 259–273. [Google Scholar] [CrossRef] - Sester, M.; Brenner, C. Continuous generalization for visualization on small mobile devices. In Proceedings of the 11th International Symposium on Spatial Data Handling (SDH’04), Leicester, UK, 23–25 August 2004; pp. 355–368. [Google Scholar]

**Figure 6.**Example of one 128px × 128px map tile from OSM (

**upper left**), at scale 1:10,000 (

**upper right**), 1:15,000 (

**bottom left**), and 1:25,000 (

**bottom right**).

**Figure 7.**Learning curves of network architectures using different loss functions at target scale 1:10,000 (

**left**), 1:15,000 (

**middle**) and 1:25,000 (

**right**) based on validation set.

**Figure 8.**Learning curves of network architectures using different optimizer at target scale 1:10,000 (

**left**), 1:15,000 (

**middle**) and 1:25,000 (

**right**) based on validation set.

**Figure 9.**Learning curves of GANs using different discriminators and weighting factors at target scale 1:10,000 (

**left**), 1:15,000 (

**middle**) and 1:25,000 (

**right**) based on validation set.

**Figure 11.**Buildings shown at original scale 1:15,000, from top to bottom: tests 1 and 2; from left to right: input image, target image, U-net prediction, residual U-net prediction, and GAN prediction.

**Figure 12.**Evaluation of different models on buildings with orientations at three target map scales.

**Figure 13.**Evaluation of different models on buildings with orientations at three target map scales.

**Figure 14.**Comparison of building map generalized with CHANGE (

**top**) and predicted representation with deep neural networks.

**Figure 15.**Comparison of building map generalized with CHANGE (

**top**) and predicted representation with deep neural networks. Visualization is at the original target scale size.

**Figure 16.**Original map scale (

**upper left**), generalization results from residual U-net at map scales 1:10,000 (

**upper right**), 1:15,000 (

**bottom left**) and 1:25,000 (

**bottom right**)—residential and industrial buildings, inner city.

Input vs. Target | U-net | Residual U-net | GAN | |
---|---|---|---|---|

1:25,000 | 2.5172% | 1.3574% | 1.1589% | 1.2942% |

1:15,000 | 1.3551% | 0.7138% | 0.4899% | 0.6137% |

1:10,000 | 0.7396% | 0.4956% | 0.3156% | 0.3536% |

**Table 2.**Precision/Recall/F1-score on the positive class at three target map scale for the independent test set.

Input vs. Target Prec. Rec. F1 | U-net Prec. Rec. F1 | Residual U-net Prec. Rec. F1 | GAN Prec. Rec. F1 | |
---|---|---|---|---|

1:25,000 | 93.94% 93.02% 93.48% | 96.42% 96.59% 96.50% | 97.19% 96.83% 97.01% | 96.76% 96.56% 96.66% |

1:15,000 | 96.84% 96.13% 96.49% | 98.19% 98.12% 98.15% | 98.82% 98.65% 98.73% | 98.55% 98.28% 98.41% |

1:10,000 | 98.20% 97.95% 98.08% | 98.77% 98.65% 98.71% | 99.20% 99.16% 99.18% | 99.14% 99.02% 99.08% |

Input vs. Target | U-net | Residual U-net | GAN | |
---|---|---|---|---|

Test 1 | 2.3587% | 1.1153% | 0.8850% | 0.9752% |

Test 2 | 1.9261% | 0.8975% | 0.7150% | 0.8191% |

**Table 4.**Precision/recall/F1-score on the positive class for independent test sets at map scale 1:15,000.

Input vs. Target Prec. Rec. F1 | U-net Prec. Rec. F1 | Residual U-net Prec. Rec. F1 | GAN Prec. Rec. F1 | |
---|---|---|---|---|

Test 1 | 94.74% 93.13% 93.93% | 97.32% 97.05% 97.19% | 97.99% 97.55% 97.77% | 97.83% 97.24% 97.54% |

Test 2 | 97.60% 95.42% 96.50% | 98.51% 98.25% 98.38% | 99.01% 98.41% 98.71% | 98.92% 98.12% 98.52% |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Feng, Y.; Thiemann, F.; Sester, M.
Learning Cartographic Building Generalization with Deep Convolutional Neural Networks. *ISPRS Int. J. Geo-Inf.* **2019**, *8*, 258.
https://doi.org/10.3390/ijgi8060258

**AMA Style**

Feng Y, Thiemann F, Sester M.
Learning Cartographic Building Generalization with Deep Convolutional Neural Networks. *ISPRS International Journal of Geo-Information*. 2019; 8(6):258.
https://doi.org/10.3390/ijgi8060258

**Chicago/Turabian Style**

Feng, Yu, Frank Thiemann, and Monika Sester.
2019. "Learning Cartographic Building Generalization with Deep Convolutional Neural Networks" *ISPRS International Journal of Geo-Information* 8, no. 6: 258.
https://doi.org/10.3390/ijgi8060258