Application of Intelligent Technology in Facade Style Recognition of Harbin Modern Architecture
Abstract
:1. Introduction
2. Literature Review
2.1. Historical Dimension Classification Method of Architectural Fields
2.2. Rational Technical Dimension of Architectural Style Classification
3. Methodology
3.1. Classification Method of Modern Harbin Architectural Facade Styles
3.2. Manual Processing Method of Image Data
3.2.1. Data Collection and Image Preprocessing
3.2.2. Data Analysis and Decorative Pattern Extraction
3.2.3. Deformation and Variation of Image Data
3.3. Classification Method of Architectural Facade Styles Based on Deep Learning
3.3.1. Residual Networks
3.3.2. Multi-Scale Residual Network Based on Channel Attention
4. Experiment and Result Analysis
4.1. Experimental Datasets
4.2. Experimental Environment and Configuration
4.3. Experimental Results and Analysis
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Hou, Y.B. Overview of Modern Chinese Architectural History (Harbin); China Architecture & Building Press: Beijing, China, 1992. [Google Scholar]
- Pan, G.X. History of Chinese Architecture, 5th ed.; China Architecture & Building Press: Beijing, China, 2014. [Google Scholar]
- Notice on Further Strengthening the Management of Urban and Architectural Styles. Available online: https://zjj.sxxz.gov.cn/xydt/hyxw/202005/t20200514_3522656.html (accessed on 22 February 2022).
- Xia, B.; Li, X.; Shi, H. Style Classification and Prediction of Residential Buildings Based on Machine Learning. J. Asian Archit. Build. Eng. 2020, 19, 714–730. [Google Scholar] [CrossRef]
- Leland, M.R.; Amanda, C.R. Understanding Architecture: Its Elements, History, and Meaning; Routledge: London, UK, 2013. [Google Scholar]
- Xu, S.B. Modern Chinese Architectural History—Westernization and Modernization of Chinese Cities and Buildings; China Architecture & Building Press: Beijing, China, 2016; Volume 1. [Google Scholar]
- China National Knowledge Internet (CNKI). Available online: https://www.cnki.net/ (accessed on 22 February 2022).
- Feng, L. National Forms of Modern Chinese Architecture; China Architecture & Building Press: Beijing, China, 2021. [Google Scholar]
- Lai, D.L. Study on Chinese Modern Architectural History; China Architecture & Building Press: Beijing, China, 2007. [Google Scholar]
- Zhang, F.H. Modern Architectural History Research in China and Modern Architectural Heritage Protection. J. HIT Soc. Sci. Ed. 2008, 10, 12–25. [Google Scholar]
- Jin, L. The Protection and Inheritance of Architecture in New China Needs Masterpieces—Thoughts on reading Zheng Shiling’ Modern Architectural Style of Shanghai. Constr. Archit. 2020, 19, 62–64. [Google Scholar]
- Liu, S.F. Modern Transformation and Mode of Urban Architecture in Harbin; China Architecture & Building Press: Beijing, China, 2003. [Google Scholar]
- Krajin, H.N. Harbin—An Ideal City in the Minds of Russian People; Harbin Publishing House: Harbin, China, 2007. [Google Scholar]
- Chang, H.S. Harbin Architectural Art; Heilongjiang Science and Technology Press: Harbin, China, 1990. [Google Scholar]
- Nie, Y.L. Harbin Historic Building (I); Harbin Urban and Rural Planning Bureau: Harbin, China, 2005.
- Owen, H. Architectural Styles: A Visual Guide; Beijing Arts and Photography Publishing House: Beijing, China, 2017. [Google Scholar]
- Shalunts, G.; Haxhimusa, Y.; Sablatni, R. Architectural Style Classifification of Building Facade Windows. In Advances in Visual Computing 6939; Springer: Las Vegas, NV, USA, 2011; pp. 280–289. [Google Scholar]
- Goel, A.; Juneja, M.; Jawahar, C.V. Are buildings only instances? Exploration in architectural style categories. In Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing, Mumbai, India, 16–19 December 2012. [Google Scholar]
- Zhao, P.; Miao, Q.; Song, J. Architectural style classification based on feature extraction module. IEEE Access 2018, 6, 52598–52606. [Google Scholar] [CrossRef]
- Mathias, M.; Martinovic, A.; Weissenberg, J.; Haegler, S.; Van Gool, L. Automatic Architectural Style Recognition. In Proceedings of the 4th ISPRS International Workshop 3D-ARCH 2011, Trento, Italy, 2–4 March 2011; Volume XXXVIII-5/W16, pp. 171–176. [Google Scholar]
- Llamas, J.; Lerones, P.; Zalama, E.; Gómez García-Bermejo, J. Applying Deep Learning Techniques to Cultural Heritage Images within the INCEPTION Project. In EuroMed 2016: Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection. Part I, Nicosia, Cyprus, 31 October–5 November 2016; Springer: Cham, Switzerland, 2016; Volume 10059, pp. 25–32. [Google Scholar]
- Llamas, J.; Lerones, P.M.; Medina, R.; Zalama, E.; Gómez-García-Bermejo, J. Classification of architectural heritage images using deep learning techniques. Appl. Sci. 2017, 7, 992. [Google Scholar] [CrossRef] [Green Version]
- Wei, W.X.; Wang, N. The Application Status and Prospect of Artificial Intelligence in the Management of Smart Scenic Spots. J. Hebei Youth Adm. Cadres Coll. 2019, 31, 54–59. [Google Scholar]
- Zhang, R.; Zhao, Y.; Kong, J. Intelligent Recognition Method of Decorative Openwork Windows with Sustainable Application for Suzhou Traditional Private Gardens in China. Sustainability 2021, 13, 8439. [Google Scholar] [CrossRef]
- Xu, H.; Wang, L.B.; Fang, Z.X. Street-Facing Architectural Image Mapping and Architectural Style Map Generation Method Using Street View Images. Geomat. Inf. Sci. Wuhan Univ. 2021, 46, 659–671. [Google Scholar]
- Yi, Y.K.; Zhang, Y.; Myung, J. House Style Recognition Using Deep Convolutional Neural Network. Autom. Constr. 2020, 118, 103307. [Google Scholar] [CrossRef]
- Yang, S.L. Retrospection of Chinese Modern Architecture. Archit. J. 1987, 3, 59–63. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G. ImageNet Classification with Deep Convolutional Neural Networks Alex. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-scale Image Recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar]
- Liu, L.; Wang, H.; Wu, C. A Machine Learning Method for the Large-scale Evaluation of Urban Visual Environment. arXiv 2016, arXiv:1608.03396. [Google Scholar]
- Sa, I.; Ge, Z.; Dayoub, F.; Upcroft, B.; Perez, T.; McCool, C. DeepFruits: A Fruit Detection System Using Deep Neural Networks. Sensors 2016, 16, 1222. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rawat, W.; Wang, Z. Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review. Neural Comput. 2017, 29, 2352–2449. [Google Scholar] [CrossRef] [PubMed]
- Chu, W.-T.; Tsai, M.-H. Visual Pattern Discovery for Architecture Image Classification and Product Image Search. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, Hong Kong, China, 5–8 June 2012. [Google Scholar]
- Zhang, L.; Song, M.; Liu, X.; Sun, L.; Chen, C.; Bu, J. Recognizing Architecture Styles by Hierarchical Sparse Coding of Blocklets. Inf. Sci. 2014, 254, 141–154. [Google Scholar] [CrossRef]
- Xu, Z.; Tao, D.; Zhang, Y.; Wu, J.; Tsoi, A.C. Architectural Style Classification Using Multinomial Latent Logistic Regression. In Computer Vision—ECCV 2014; Springer: Cham, Switzerland, 2014; Volume 8689, pp. 600–615. [Google Scholar]
- Yu, F.; Koltun, V. Multi-scale Context Aggregation by Dilated Convolutions. arXiv 2015, arXiv:1511.07122. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 7132–7141. [Google Scholar]
- Park, Y.J.; Tuxworth, G.; Zhou, J. Insect Classification Using Squeeze-and-excitation and Attention Modules-a Benchmark Study. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–29 September 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 3437–3441. [Google Scholar]
- Bengio, Y. Deep Learning of Representations for Unsupervised and Transfer Learning. In Proceedings of the ICML Workshop on Unsupervised and Transfer Learning, Bellevue, WA, USA, 2 July 2011; Volume 27, pp. 17–36. [Google Scholar]
- Ninomiya, H. Neural Network Training Based on Quasi-Newton Method Using Nesterov’s Accelerated Gradient. In Proceedings of the IEEE Region 10 Conference, Singapore, 22–25 November 2016; pp. 51–54. [Google Scholar]
- Zhou, B.L.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, 26–30 June 2016; pp. 2921–2929. [Google Scholar]
Style | Roof\Dome | Eave | Window | Porch | Balcony | Column | Cornice | Decorative Pattern | Parapet Wall | Dormer Window |
---|---|---|---|---|---|---|---|---|---|---|
Classical | √ | √ | √ | √ | √ | |||||
Eclectic | √ | √ | √ | √ | √ | √ | √ | √ | ||
New art movement | √ | √ | √ | √ | √ | √ | ||||
Decorative art | √ | |||||||||
Gothic | √ | √ | √ | √ | ||||||
Renaissance | √ | √ | √ | √ | √ | √ | √ | √ | √ | |
Byzantine | √ | √ | √ | √ | ||||||
Baroque | √ | √ | √ | √ | √ | √ | √ | √ | √ | |
Russian Style | √ | √ | √ | |||||||
Japanese Style | ||||||||||
Imitation Chinese traditional | √ | √ | √ | |||||||
Chinese–Western integration | √ | √ | √ | √ | √ | √ | √ |
Harbin Tang Gong Pavilion | |||
---|---|---|---|
Restored facade drawing | |||
Reference photos | Old photo | Photo before restoration | Photo after restoration |
Deformation of Windows | Deformation of Rails | Deformation of Facades |
---|---|---|
Category | Example | ||||
---|---|---|---|---|---|
Chinese–Western integration architecture | Facade | ||||
Details | |||||
CAD drawing | |||||
Russian style architecture | Facade | ||||
Details | |||||
CAD drawing | |||||
Eclectic architecture | Facade | ||||
Details | |||||
CAD drawing | |||||
Renaissance architecture | Facade | ||||
Details | |||||
CAD drawing | |||||
New art movement architecture | Facade | ||||
Details | |||||
CAD drawing |
Layers | Output Size | ResNet (50 Layer) | CA-MSResNet (50 Layer) |
---|---|---|---|
Conv 1 | 112 × 112 | 7 × 7, 64, stride 2 | |
Stage 1 | 56 × 56 | 3 × 3 max pool, stride 2 | |
Stage 2 | 28 × 28 | ||
Stage 3 | 14 × 14 | ||
Stage 4 | 7 × 7 | ||
Classification On Layer | 1 × 1 | 7 × 7 global average pool 12-d FC, Softmax |
Models | Accuracy | F1-Score |
---|---|---|
AlexNet [28] | 0.703 | 0.655 |
VGG [29] | 0.773 | 0.728 |
GoogLeNet [30] | 0.786 | 0.753 |
ResNet [31] | 0.798 | 0.777 |
CA-MSResNet-M1 (only trained by architectural facades dataset) | 0.836 | 0.822 |
CA-MSResNet-M2 (Enhanced training model using architectural details dataset) | 0.859 | 0.862 |
CA-MSResNet-M3 (Enhanced training model using architectural details dataset and CAD drawing dataset) | 0.875 | 0.902 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shan, L.; Zhang, L. Application of Intelligent Technology in Facade Style Recognition of Harbin Modern Architecture. Sustainability 2022, 14, 7073. https://doi.org/10.3390/su14127073
Shan L, Zhang L. Application of Intelligent Technology in Facade Style Recognition of Harbin Modern Architecture. Sustainability. 2022; 14(12):7073. https://doi.org/10.3390/su14127073
Chicago/Turabian StyleShan, Linlin, and Long Zhang. 2022. "Application of Intelligent Technology in Facade Style Recognition of Harbin Modern Architecture" Sustainability 14, no. 12: 7073. https://doi.org/10.3390/su14127073