Damage Classification in Historical Buildings Through Transfer Learning Approaches

Abstract

Historical buildings are important cultural assets that reflect the identity of cities and preserve the collective memory of societies. However, these structures are increasingly exposed to environmental degradation and human-induced impacts, making their systematic documentation and condition assessment essential for effective conservation strategies. Recent advances in artificial intelligence have provided powerful tools for image-based analysis in the field of heritage preservation. In particular, transfer learning enables the adaptation of pre-trained deep learning models to domain-specific tasks with limited labeled data. In this study, a deep transfer learning-based framework is proposed for automatic damage detection and classification in historical buildings. A new near-balanced dataset of 20,000 images spanning six deterioration categories was developed and made publicly available. Ten convolutional neural network and transformer architectures pre-trained on ImageNet were systematically compared under a unified Bayesian optimization protocol. Experimental results on a held-out test set show that EfficientNetB3 achieves the highest classification accuracy (97.65%), while AlexNet obtains the lowest performance (83.89%); the validation set was used exclusively for hyperparameter tuning. The results demonstrate that transfer learning-based models can effectively identify visually observable deterioration patterns and provide reliable support for automated documentation processes. The proposed framework contributes to the development of data-driven decision-support tools for digital documentation and condition assessment in heritage conservation.

1. Introduction

Cities shaped by the interaction of physical, cultural, and social components are complex multilayered systems [1,2]. Urban identity extends beyond the built environment and arises from the integration of natural features, social norms, and cultural aspects [1] Historical buildings, as tangible examples of urban memory, are important not only for their physical existence but also for their role in collective memory. Thus, these structures are playing a key role in preserving urban identity [1,3,4]. Therefore, in order to protect this identity, it is necessary to design plans and interventions carefully [1].

Historical environments consist of both monumental structures and a coherent spatial arrangement bearing tangible cultural traces. Beyond their physical presence, these settings provide insight into the everyday practices, value systems, and construction knowledge of past societies [5,6,7]. Over time, many of these structures have been subject to successive modifications driven by changing functional requirements, social conditions, and political contexts. As a result, historical buildings should be understood as dynamic cultural heritage assets that document long-term socio-economic and political transformations rather than as static remnants of the past [6,8,9]. In current practice, restoration and reinforcement efforts implemented in line with increasing conservation awareness play a strategic role in ensuring the continuity of these historical assets [8]. These interventions, require a multidisciplinary approach that focuses not only on aesthetic considerations but also on structural safety [10]. In this context, interventions that preserve the original texture through appropriate technical solutions have become a widely accepted principle in conservation-oriented approaches [10,11].

Accurate identification of existing damage constitutes a prerequisite for informed conservation decisions in historic buildings. Damage types often differ according to material behavior and environmental exposure. Overlooking these differences may result in inappropriate or structurally incompatible repair decisions. For this reason, systematic damage classification plays a critical role in guiding conservation interventions.

Historical structures are continuously exposed to solar radiation, temperature variations, humidity, precipitation, and freeze–thaw cycles. Over time, these environmental processes contribute to the development of structural damage. However, wars, environmental pollution, and other human-induced interventions also contribute to structural deterioration. These deteriorations can manifest as material loss, cracks, settlements due to ground movements, or biological and chemical effects [11]. Methods used to determine the damage that has occurred are generally evaluated under two main headings: destructive and non-destructive. Non-destructive techniques are widely preferred because of their ability to be applied without jeopardizing the structure, but they generally provide limited data. Therefore, expert opinion is often required to make a final decision. In contrast, destructive methods generally provide clearer results but have disadvantages such as negatively affecting structural integrity and requiring special laboratory conditions, advanced equipment, and qualified personnel.

Data-driven approaches are now widely used in studies dealing with historic structures. Visual information obtained from these structures can now be processed more efficiently through artificial intelligence-based methods [12]. In particular, machine learning and deep learning techniques support accurate and rapid analyses through feature extraction and classification from large datasets in conservation and restoration studies [13]. Here, the success of the method depends on the creation of significant and balanced datasets along with high computational resources. In scenarios where these conditions cannot be met, transfer learning methods become particularly advantageous [14]. In the case of limited datasets, successful results can be obtained by adapting pre-trained models. This reduces both training time and computational costs, while facilitating the more effective application of deep learning in the field of historical building conservation [15].

Prior studies on image-based damage detection and deterioration recognition in historic structures are summarized below. Since these studies address different tasks—including classification, object detection, segmentation, and, in a limited number of cases, multimodal sensing—the reported performance values correspond to different evaluation measures, such as classification accuracy, F1-score, pixel-level mIoU, and average precision, rather than modal identification accuracy or physics-based damage-prediction error. With the exception of BDD-Net [16], which combines optical and LiDAR data, the cited image-based approaches rely primarily on visual features extracted from photographs or remote-sensing imagery and do not incorporate soil–structure interaction parameters. In contrast, vibration-based structural health monitoring (SHM) and nonlinear simulation studies represent complementary model-based research streams and should not be considered directly comparable to image-based approaches. Guzel et al. [17] reported 85% accuracy and a 91% F1 score when using a ResNet-34 model on satellite imagery for building damage analysis. Using drone-based images, Maras and Sariyildiz [18] achieved an accuracy of 95.1% with a Mask R-CNN framework. Seydi et al. [16] reached over 88% accuracy using the BDD-Net model, which combines optical and LiDAR data. Maskeliunas et al. [19] reported 98.41% accuracy in classifying architectural facade styles from UAV images. Fu and Angkawisittpan [20] detected surface defects with mIoU above 92% and 95.78% accuracy using Swin Transformer and YOLOv5 models. Sasithradevi et al. [21] classified historic building silhouettes with 95.5% accuracy using the MonuNet model. Hatir et al. [22] identified weathering damage in stone structures with 99.4% accuracy using deep learning techniques. Mansuri and Patel [23] achieved 91.58% accuracy in historic structures using Faster R-CNN. Maroun et al. [24] reported 95% accuracy in post-explosion crack detection. Wang et al. [25] achieved an average precision of 95% using Faster R-CNN with the ResNet101 architecture. Hatir et al. [26] reached 98.18% accuracy in the Gumusler archaeological site. Reis and Khoshelham [27] demonstrated higher crack-detection accuracy than existing methods using the ReCRNet model. Unal [28] reported high classification accuracy in detecting damage in historic stone structures with Mask R-CNN. Bingol et al. [29] identified structural irregularities with over 90% accuracy. Karacay [30] detected reinforced concrete cracks with 92.79 -96.41% accuracy across different datasets using a U-Net-based model. Ustaoglu [31] classified building physics issues with 95.21% accuracy using CNN. Finally, Ararat [32] developed a high-performing multi-class damage classification model capable of distinguishing different types of damage in historic structures. Complementary structural health monitoring studies on steel truss bridges have investigated strain-based sensitivity analysis for optimal sensor placement [33] and partial-model damage identification based on stiffness separation for long-span trusses [34], offering model-based alternatives to the image classification approaches summarized above. Internationally, energy-based seismic retrofit design for historic churches through nonlinear modelling [35], critical reviews of nonlinear seismic response in unreinforced masonry heritage [36], and seismic vulnerability analyses that link earthquake-induced cracking to strengthening options [37] further contextualize structural conservation needs that façade-level image screening can help triage but cannot replace. Broader heritage monitoring and preservation frameworks—including model updating of cultural heritage buildings and long-term monitoring-data cleaning for in-service masonry—underscore the need to embed image-based classifiers within established conservation workflows rather than treating them as stand-alone tools [38,39].

Existing heritage-imaging studies are often limited by restricted damage diversity and relatively small datasets. To address these limitations, we developed a new near-balanced corpus comprising 20,000 high-resolution images organized into six classes, including five deterioration categories—biological and chemical degradation, cracks, human-induced damage, and material loss—as well as undamaged surfaces (publicly released as [40]). Furthermore, ten transfer-learning architectures were systematically evaluated using a unified Bayesian optimization protocol.

The study evaluates whether ImageNet-pre-trained models can reliably classify visually observable deterioration on historic façades and identifies which architectural families perform best under identical data and tuning conditions [41]. The findings are intended to support digital documentation in heritage conservation and to provide a comparative reference for future image-based damage classification research.

2. Deterioration Processes in Historic Structures

Historic buildings are exposed to various forms of deterioration over time. Environmental exposure and external actions can impair structural elements through coupled physical and chemical mechanisms [28,42]. Because deterioration processes interact, information on the structural system, materials, and environmental context is required when planning interventions [43] (Figure 1).

Figure 1. The process of damage to buildings [43].

3. Types of Damage in Historic Buildings

Natural disasters such as earthquakes, landslides, and floods are among the most serious threats that cause structural damage. Earthquakes weaken the integrity of structural systems through seismic waves, floods affect foundations through changes in groundwater levels, and rapid currents cause damage to infrastructure elements such as bridges [28,44]. Numerical studies of unreinforced masonry under seismic loading further emphasize that visible wall cracking must be interpreted together with nonlinear capacity models and evaluated strengthening techniques [36,37]. In addition to the effects of disasters, anthropogenic environmental pollutants, particularly emissions from transportation and heating, cause the formation of dirt layers on the surfaces of monumental structures and surface losses in stone or masonry elements. Design and application errors, such as inadequate dimensioning, use of low-quality materials, inappropriate joint details, incorrect restoration practices, and poor workmanship, weaken the load-bearing capacity [28].

Furthermore, heavy vehicle traffic in historic urban areas accelerates the damage process by creating vibration-induced dynamic effects [28]. Systematic damage classification supports conservation planning [28,41,45]; the principal categories, symptoms, and causes relevant to historic buildings are summarized in Table 1. Table 1 presents the broader conservation taxonomy discussed in the literature; the six-class dataset used for model training operationalizes mechanical damage predominantly as visible cracks (Section 5.1).

Table 1. Common damage patterns in historic buildings [6,44,46,47,48,49,50,51,52,53].

Type of Damage	Subtypes/Symptoms	Causes	Effects
Structural damages	Wall cracks; roof covering collapse; damage to load-bearing elements; foundation settlement; impacts; material fractures; crushing/buckling	Ground movements; weakness of the load-bearing system; overloading; human interference; natural disasters (earthquake, fire, etc.); traffic/vibration effects	Reduction in load-bearing capacity; risk of collapse and deformation; local failures; decrease in structural stability
Biological damages	Mold, moss, fungus; plant/ivy infestation; insect attack; animal residues	High humidity; lack of maintenance; accumulation of organic matter	Aesthetic deterioration; physical damage; reduction in load-bearing capacity
Human-induced damages	Faulty restoration; inappropriate materials; unqualified interventions	Unconscious applications; new spatial requirements; vandalism	Loss of authenticity; unpredictable damage
Surface damages	Peeling of plaster/paint; loss of decorative elements; hairline cracks	Air pollution; aging; water leakage; temperature differences	Loss of aesthetic value; failure of protective layers
Chemical damages	Corrosion (metals); abrasion; freeze–thaw cracks; porosity and efflorescence; soot blackening; salt crystallization	Moisture contacting metal; acidic environments; climate; air pollution; acid rain	Weakening of structural elements; surface deterioration; loss of durability

4. Traditional Damage Detection Methods for Historic Structures

Systematic condition assessment precedes conservation interventions in historic buildings [24]. Visual inspection remains common, but complementary non-destructive and, where authorized, destructive tests are also used [23]. Traditional approaches are grouped as destructive and non-destructive methods.

Non-destructive testing methods enable the assessment of the functionality and safety of structures or materials without causing physical damage. While material quality and construction standards are the primary focus in new buildings, non-destructive methods are applied in historic structures mainly to assess structural integrity and load-bearing capacity. The ability to apply them to the same structure at different times provides advantages in monitoring structural changes and identifying potential risks in advance [25]. Non-destructive testing methods offer advantages such as low cost, ease of application, and reduced need for repairs; however, they also have disadvantages such as sensitivity to environmental conditions, limitations in detecting serious damage, and the high cost of some equipment. The characteristics of the structure play a decisive role in method selection. Techniques such as ultrasonic measurement [11], hardness measurement, radioactive methods, infrared thermography [54], in-situ pressure and shear tests, endoscopic and ground-penetrating radar techniques [55], and the liquid impregnation method [56] can be listed as non-destructive methods that can be applied to historic buildings.

Non-destructive methods, which are limited in terms of data accuracy, provide information without damaging the structure. However, in some cases, complementary destructive tests may be performed to clearly determine the characteristic properties of the material. Tests are conducted on samples taken from structural elements to determine their mechanical and physical properties. In this context, for example, the capillary water absorption capacities, water absorption rates by weight, and unit weights of porous volumes of stone, brick, and mortar samples can be determined [8]. Destructive methods have the advantage of providing direct observation and reliable results, but they also have disadvantages such as high cost, time loss, and material loss. Physical and mechanical tests performed on core samples taken appropriately from historical structures with the permission of Conservation Area Committees can be evaluated in this context. In some cases, hybrid use with non-destructive methods may be preferred [8].

Conventional destructive and non-destructive techniques have long been used to evaluate damage in historic buildings. More recently, data-driven approaches have begun to attract attention, particularly in studies dealing with complex and visually diverse deterioration patterns. Image-based deep learning methods are able to capture such patterns directly from large collections of visual data, without relying on predefined damage descriptors. This approach supports a more consistent evaluation framework and helps reduce variability associated with expert-based assessments. Within this framework, the present study explores the use of selected transfer learning models for damage classification in historical structures.

5. Materials and Methods

In this study, the applicability of transfer learning-based deep learning methods for image-based crack detection and surface-deterioration classification in historic structures was evaluated to support their preservation and restoration. The near-balanced image corpus, class definitions, acquisition protocol, and preprocessing steps are described in Section 5.1; the subsections below document model selection, hyperparameter optimization, and evaluation metrics.

Ten different pre-trained deep learning models—EfficientNetB3, Vision Transformer (ViT), XceptionNet, InceptionNetV3, AlexNet, VGG-19, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt-50—were evaluated using the transfer learning approach. They were selected deliberately—not to crown a single architecture, but to compare distinct design paradigms under identical data, splitting, and Bayesian optimization settings. The ten networks span five complementary families: classical CNN baselines (AlexNet, VGG-19); lightweight models suited to field or mobile deployment (MobileNetV2, ShuffleNetV2); architectures emphasizing feature reuse and residual depth (DenseNet121, ResNeXt-50); modern high-capacity CNNs with compound scaling or depthwise factorization (EfficientNetB3, XceptionNet, InceptionNetV3); and a transformer encoder (ViT) for long-range façade context. All use publicly available ImageNet weights implemented in standard deep-learning libraries, which supports reproducibility and alignment with prior heritage-imaging studies. Additionally, eight different hyperparameters were optimized using Bayesian optimization to determine their optimal values. Details regarding the dataset, methodology, models, and hyperparameter optimization procedures are presented in the following subsections.

5.1. Dataset Preparation

The dataset was constructed from images of historical buildings representing different periods and material types (e.g., adobe, brick, wood, stone, and metal). Images originated from heritage structures in Türkiye and several European countries, with substantial coverage from northwestern Türkiye (including Istanbul) and from eastern and southeastern Anatolian provinces (e.g., Elazığ, Malatya, Mardin, Şanlıurfa, Diyarbakır). Data were gathered between 2021 and 2025 through field surveys conducted within restoration-studio courses, in-class applications, the lead author’s personal archive, and supplementary submissions from students enrolled in the same instructional program. Photographs were taken with compact digital cameras and smartphones under uncontrolled outdoor conditions; EXIF metadata were not used as model inputs. All models were trained exclusively on RGB photographs. Material properties, mechanical parameters, geometric measurements, structural-analysis outputs, sensor data, and acquisition metadata were not used as inputs. During dataset construction, whether individual buildings had undergone prior restoration or structural retrofitting was neither recorded, distinguished, nor used as a selection or labeling criterion; therefore, the intervention history of individual structures was not considered. The framework targets image-based deterioration classification from visual evidence rather than structural health assessment, and future datasets that explicitly annotate intervention history could enable stratified evaluation. Annotation was carried out by the authors, who specialize in architecture, building physics, and architectural conservation, using the Roboflow labeling environment. Six labels were defined: biological deterioration, chemical deterioration, cracks, human-induced damage, material loss, and undamaged surfaces. A double-check workflow was applied so that labels were verified before final acceptance. Borderline cases, especially between chemical versus biological surface change, and between human-induced interventions versus material loss, were resolved through consensus before a final class assignment. The classes were distributed in a near-balanced manner: each damage class contains 3400 images, while the undamaged (reference) class includes 3000 images (Table 2; approximately 11.8% fewer reference samples). On the held-out test split, this mild undersampling did not produce an observable penalty for undamaged-surface recall in the best-performing models; confusion-matrix analysis in Section 6 indicates that remaining errors arose primarily between visually similar damage categories rather than between damaged and undamaged surfaces. Sample images are presented in Figure 2, and class distributions are provided in Table 2.

Table 2. Distribution of images across different classes in the dataset [40].

Class Name	Number of Images
Biological deterioration	3400
Chemical deterioration	3400
Cracks	3400
Human-induced damage	3400
Material loss	3400
Undamaged	3000

Figure 2. Sample images from different classes in the dataset.

Images obtained from different devices and field conditions exhibit heterogeneity in terms of resolution and aspect ratio. Therefore, all images were resized to 224 × 224 pixels to ensure compatibility with the input layers of the deep learning models and normalized to the RGB color space. Bilinear resizing was applied as a standard step compatible with fixed-size CNN and ViT inputs. We acknowledge that bilinear resizing to 224 × 224 pixels may reduce the visibility of extremely fine cracks and localized efflorescence patterns. This resolution was chosen primarily to ensure compatibility and fair comparison among all evaluated architectures under identical experimental conditions. Although a dedicated quantitative ablation using multiple input resolutions was beyond the scope of this study, the six deterioration categories are characterized not only by micro-scale detail but also by texture, colour distribution, surface morphology, and contextual information; the high test accuracies achieved by EfficientNetB3 (97.65%) and MobileNetV2 (96.80%) indicate that sufficient discriminative information was retained for reliable classification. This limitation is explicitly acknowledged in Section 7; future studies incorporating higher input resolutions (e.g., 384 × 384 and 512 × 512), patch-based analysis, and multi-scale learning strategies are expected to provide a more rigorous assessment of the influence of image resizing on subtle deterioration features. The resizing procedure was applied carefully to retain the essential visual characteristics of the structures. This step also helped standardize the input data and improve training efficiency. Consequently, this near-balanced and original dataset, created by the authors and unparalleled in the literature, serves as a reliable reference for evaluating the performance of deep learning-based transfer learning approaches in classifying damage types in historical structures.

5.2. Dataset Splitting

70% of the dataset was used in the model training phase. The 20% allocated validation set facilitated reliable optimization of hyperparameters, while the remaining 10%, previously unseen by the models, was evaluated in the testing phase. This data subdivision limits the risk of overfitting and creates a comparable reference frame for all models. Splits were stratified by class and drawn from the same acquisition pool; independent external validation on another heritage corpus is left to future work and is discussed below. Partitioning was performed at the individual-image level using stratified random assignment. Duplicate images were excluded before splitting, and no deliberate placement of photographs from the same acquisition context across training, validation, and test subsets was performed. However, because consistent building- or façade-level identifiers were not recorded for all images, structure-wise partitioning could not be strictly enforced; a limited degree of spatial correlation between subsets therefore cannot be completely excluded despite the geographic, material, and acquisition diversity described in Section 5.1. Table 3 summarizes the distribution of the dataset according to damage classes.

Table 3. Dataset composition and class-wise distribution across subsets.

Subset	Bio.	Chem.	Cracks	Human-Ind.	Mat. Loss	Undam.
Training	2373	2373	2387	2372	2379	2119
Validation	690	682	683	680	676	587
Test	337	345	330	348	345	294
Total	3400	3400	3400	3400	3400	3000

Beyond the Bayesian-optimized settings, aggressive data augmentation was avoided so as not to distort subtle deterioration appearance. To mitigate overfitting, early stopping was applied with validation loss as the monitored metric and a patience of ten epochs; training halted automatically when validation loss ceased to improve. The validation split therefore informed hyperparameter search and early stopping, but was not used to draw final comparative conclusions; those rely on the independent 10% test split (Section 6). Bayesian optimization maximized validation accuracy when searching hyperparameters.

5.3. Transfer Learning Models Employed

Ten pre-trained deep learning models were evaluated for classifying damages in historical buildings using a transfer learning approach. Nine models are CNN-based and one utilizes a transformer architecture, including EfficientNetB3, Vision Transformer (ViT), XceptionNet, InceptionNetV3, AlexNet, VGG-19, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt50. Initialization used ImageNet ILSVRC-2012 weights exposed through Keras Applications (and the ViT variant available in the same ecosystem). Transfer learning was implemented using a feature-extraction-based strategy. All pre-trained base-model layers were frozen throughout training, and only the newly attached classification head—a fully connected layer with six logits, softmax activation, and dropout regularized within the 0.1–0.5 interval selected during Bayesian optimization—was trained on the target dataset. This approach allows the models to leverage low-level and mid-level feature representations learned from large-scale ImageNet pre-training, while adapting the final classification layer to the specific task of damage classification in historical buildings. The total number of training epochs for each architecture was determined via Bayesian optimization within the range of 10–35. Categorical cross-entropy served as the default objective for the softmax head; during Bayesian optimization, Kullback–Leibler divergence was additionally allowed for specific architectures. For AlexNet, VGG-19, and XceptionNet, BO selected KL divergence because these architectures tended to yield overconfident predictions on heterogeneous heritage textures. KL divergence penalizes mismatches between predicted and target class probabilities, improving calibration and stabilizing training of the classification head; deeper architectures (e.g., DenseNet121, ViT) converged more reliably with categorical cross-entropy. KL was not imposed a priori—it emerged as the optimal loss for these three models in the search. The comparison includes lightweight models like MobileNetV2 and ShuffleNetV2, traditional CNNs such as AlexNet and VGG-19, more advanced architectures like DenseNet121 and ResNeXt-50, a scaled model (EfficientNetB3), and a transformer-based approach (ViT). Older models were intentionally kept as clear, interpretable baselines to help highlight the improvements brought by newer architectures under the same data and tuning conditions, rather than to compete directly in terms of performance. All models were trained on a custom dataset designed for damage detection. Table 4 provides an overview of the deep learning models employed in this study, including their architecture, key design features, and notable properties.

Table 4. Deep learning models utilized in this study.

Model	Definition and Role in This Work
EfficientNetB3	EfficientNet balances depth, width, and resolution via compound scaling with strong accuracy and efficiency across B0–B7 scales [57,58]; EfficientNetB3 retains representative capacity for heritage textures while limiting parameter overhead [59,60].
Vision Transformer (ViT)	Processes image patches as a sequence with self-attention for global context [61], supporting long-range dependencies compared with purely local CNN filters [62,63,64].
XceptionNet	Depthwise separable convolutions replace Inception blocks [65], deepening feature extraction (36 conv. layers) with strong ImageNet baselines [66,67].
InceptionNet (V3)	Multi-branch Inception family widely used for transfer learning [68]; InceptionV3 stacks convolutions, pooling, concatenation, dropout, and softmax layers [69].
AlexNet	Historic ILSVRC breakthrough architecture with five convolution/pooling stages and three dense layers [70,71,72].
VGG-19	Very deep 3 × 3 stacking for high-resolution inputs [59,73]; strong baseline for texture-sensitive façades [74,75].
ShuffleNetV2	Efficiency-oriented design for mobile-grade throughput [76,77].
DenseNet121	Dense connections reuse features and stabilize gradients [78,79]; effective on large-scale classification [59,80].
MobileNet (V2)	Lightweight depthwise separable or inverted-bottleneck stacks for embedded deployment [81,82,83,84,85].
ResNeXt50	Residual blocks with grouped convolutions and tunable cardinality [86,87].

For the applications within the scope of this study, Google Colaboratory (Colab), a cloud-based computing platform that provides GPU resources for computationally intensive tasks in artificial intelligence and deep learning, was used. Programming was performed in Python 3.11 via the Google Colab notebook interface, with datasets and labels stored in Google Drive. For model creation and training, TensorFlow 2.18.x and the Keras 3.x API were used [88,89,90]. The training durations reported in this study should be interpreted as approximate and indicative values rather than standardized or reproducible hardware benchmarks. Experiments were conducted in the Google Colab Pro+ environment using NVIDIA Tesla T4 and NVIDIA A100 GPUs (NVIDIA Corporation, Santa Clara, CA, USA), where GPU allocation and available memory may vary between sessions. The approximate training durations summarized in Section 6 are therefore provided only to give a general sense of the computational effort required under practical research conditions, rather than to support strict performance comparisons between architectures. The complete labelled image dataset is publicly available through IEEE DataPort (DOI: 10.21227/j8j6-wp24) [40], enabling independent retraining under the stratified 70/20/10 protocol reported in Table 3. To support reproducibility, the manuscript specifies preprocessing (224 × 224 bilinear resize, RGB normalization), architecture definitions, the Bayesian optimization protocol, optimized hyperparameters, and the Colab GPU environment (Section 6 and Section 7). Training notebooks and implementation scripts are not distributed as supplementary files with this article; they are available from the corresponding author upon reasonable request.

5.4. Hyperparameters Optimization Strategy

Hyperparameter optimization was incorporated into the training process to improve model performance. In the early phase of the methodology design, commonly adopted approaches such as manual tuning, grid search, and random search were taken into consideration [91]. Due to their limitations in efficiently exploring high-dimensional search spaces, Bayesian Optimization (BO) was selected as the primary optimization method. BO employs a probabilistic framework to guide the search process through an acquisition function, enabling effective parameter selection with a limited number of evaluations [92,93].

In this study, eight hyperparameters were optimized during the BO process [94]. The search space included epochs (10–35), batch size (16–64), learning rate (from $1\times {10}^{-5}$ to $5\times {10}^{-4}$), dropout (0.1–0.5), and weight decay ($1\times {10}^{-6}$ to $1\times {10}^{-3}$). In addition, categorical options were considered for the optimizer (Adam, RMSprop, SGD), loss function (categorical cross-entropy, Kullback–Leibler divergence), and data shuffling (enabled or disabled). Representative optimal configurations for ViT, DenseNet121, and MobileNetV2 are reported in Section 6. The parameters were sequentially adjusted to improve model adaptability and generalization capability for damage classification in historical structures. The overall optimization workflow is illustrated in Figure 3.

Figure 3. Schematic representation of the transfer learning process with hyperparameter tuning.

5.5. Model Performance Criteria

To assess the classification results obtained in this study, multiple performance indicators were considered. These metrics were selected to reflect both overall prediction accuracy and class-wise performance characteristics. The confusion matrix, accuracy, precision, recall, and F1 score were therefore used. Throughout this manuscript, “accuracy” refers specifically to multi-class image classification performance on the held-out test split (Table 5) and should not be interpreted as modal identification accuracy, structural response prediction, or instrumented SHM performance. A detailed description of each metric and its mathematical formulation is provided in Table 5.

Table 5. Performance metrics considered in this study.

Metric	Definition	Role/Formula
Confusion matrix	Summarizes correct/incorrect predictions through TP, TN, FP, FN counts across classes [95].	Basis for deriving scalar scores below.
Accuracy	Fraction of correctly classified instances among all predictions [96].	Overall score when classes are roughly balanced.
Precision	Fraction of predicted positives that are true positives [96].	Highlights false-positive sensitivity.
Recall (sensitivity)	Fraction of actual positives retrieved [96].	Highlights false-negative sensitivity.
F1 score	Harmonic mean of precision and recall [96].	Balances precision/recall on mildly imbalanced data.

6. Results and Analysis

Using the dataset described in Section 5.1, the ten architectures introduced in Section 5 were evaluated under the protocol detailed therein. Validation accuracy guided Bayesian optimization and early stopping, but architectural comparison and the conclusions of this study are based on the held-out test split (accuracy, precision, recall, F1-score, and confusion matrices) to limit optimism from tuning on the same validation fold.

Bayesian Optimization was applied during model training to tune eight selected hyperparameters. This approach allowed the search process to focus on promising regions of the parameter space while limiting the number of experimental trials. As a result, the optimization procedure supported efficient model training without excessive computational cost. For each model, the Bayesian optimization process was carried out over 30 iterations to identify hyperparameter configurations that maximized validation accuracy during the search. Table 6 presents the resulting configurations for all ten architectures; the best validation accuracy achieved during tuning for each architecture is reported in Table 7 but is not used for final model ranking.

Table 6. Bayesian-optimized hyperparameters for ten transfer learning models.

Model	Batch	Epoch	Optimizer	Loss	Shuffle	LR	Dropout	Wt. Decay
EfficientNetB3	32	13	RMSprop	Cross-entropy	True	$1.91\times {10}^{-5}$	0.208	$7.23\times {10}^{-5}$
ViT	32	25	Adam	Cross-entropy	True	$3.42\times {10}^{-5}$	0.240	$8.57\times {10}^{-6}$
XceptionNet	32	12	Adam	KL divergence	True	$4.00\times {10}^{-3}$	0.439	$7.18\times {10}^{-5}$
InceptionNet	16	32	Adam	Categorical CE	True	$3.00\times {10}^{-4}$	0.203	$7.28\times {10}^{-6}$
AlexNet	64	28	Adam	KL divergence	True	$1.50\times {10}^{-5}$	0.395	$1.09\times {10}^{-6}$
VGG-19	16	26	Adam	KL divergence	False	$5.00\times {10}^{-4}$	0.268	$6.74\times {10}^{-5}$
ShuffleNetV2	32	32	Adam	Cross-entropy	True	$6.28\times {10}^{-5}$	0.416	$4.22\times {10}^{-6}$
DenseNet121	64	35	SGD	Cross-entropy	True	$1.70\times {10}^{-3}$	0.100	$1.00\times {10}^{-4}$
MobileNetV2	32	29	Adam	Cross-entropy	True	$2.00\times {10}^{-4}$	0.254	$5.57\times {10}^{-6}$
ResNeXt-50	32	35	RMSprop	Cross-entropy	False	$6.01\times {10}^{-5}$	0.384	$1.00\times {10}^{-4}$

Table 7. Training duration and classification performance of models retrained with Bayesian-optimized hyperparameters.

Model	Duration	Best Val. (%)	Accuracy (%)	Prec. (%)	Recall (%)	F1 (%)
EfficientNetB3	18 h 13 min	96.50	97.65	97.69	97.59	97.63
ViT	18 h 40 min	97.57	97.15	97.22	97.06	97.11
XceptionNet	10 h 20 min	88.36	87.89	88.01	87.89	87.89
InceptionNet	14 h 35 min	89.84	90.10	90.11	90.17	90.11
AlexNet	13 h 25 min	83.69	83.89	84.60	83.85	84.00
VGG-19	10 h 52 min	92.04	90.25	90.97	89.98	90.24
ShuffleNetV2	13 h 40 min	96.74	96.65	96.62	96.64	96.62
DenseNet121	11 h 02 min	95.65	97.45	97.41	97.44	97.42
MobileNetV2	12 h 13 min	97.07	96.80	96.76	96.77	96.76
ResNeXt-50	15 h 27 min	95.11	94.70	95.19	94.73	94.83

Across the evaluated models, the batch size requirement differed substantially depending on the network design. Several architectures—namely EfficientNetB3, ViT, XceptionNet, ShuffleNetV2, MobileNetV2, and ResNeXt-50—showed their best performance when trained using batches of 32 samples. By contrast, InceptionNet and VGG-19 benefited from a more conservative batch size of 16, whereas AlexNet and DenseNet121 exhibited improved performance only when larger batches were employed, with 64 providing the most favorable results.

A similar level of variation was identified in relation to training duration. DenseNet121 and ResNeXt-50 reached their peak performance at 35 epochs, while XceptionNet and EfficientNetB3 attained high accuracy much earlier, converging after only 13 epochs. Taken together, these results demonstrate that model architectures differ considerably in their learning behavior, with some networks achieving effective training within shorter optimization cycles.

Across the evaluated experiments, higher classification accuracy was consistently associated with the use of the Adam optimizer rather than RMSprop or SGD. Specifically, Adam produced superior results in seven of the evaluated architectures, indicating a more stable convergence behavior across a broad range of network designs, including ViT, XceptionNet, InceptionNet, AlexNet, VGG-19, ShuffleNetV2, and MobileNetV2. Loss function behavior further reflected architecture-dependent performance differences. For the majority of models, categorical cross-entropy yielded the most reliable outcomes, particularly for EfficientNetB3, ViT, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt-50. Conversely, AlexNet, VGG-19, and XceptionNet demonstrated improved validation-tuning accuracy when trained using Kullback–Leibler divergence, consistent with the calibration benefits discussed in Section 5.3.

Other hyperparameters exhibited model-specific variations. The shuffle parameter was set to True for most models, whereas it was False only for VGG-19 and ResNeXt-50. The lowest learning rates were observed in AlexNet ($1.50\times {10}^{-5}$) and ViT ($3.42\times {10}^{-5}$), while the highest dropout rates were recorded in AlexNet (0.3953) and XceptionNet (0.4389). These findings indicate that model-specific differences in hyperparameter settings directly influence classification performance (Table 6).

Figure 4 and Figure 5 present training and validation accuracy and loss curves obtained during retraining with the Bayesian-optimized hyperparameters. Figure 4 shows EfficientNetB3, Vision Transformer (ViT), XceptionNet, InceptionNetV3, and AlexNet; Figure 5 shows VGG-19, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt-50. The curves highlight differences in convergence speed and stability across architectures.

Figure 4. Training and validation accuracy and loss curves for selected models (EfficientNetB3, ViT, XceptionNet, InceptionNet, and AlexNet).

Figure 5. Training and validation accuracy and loss curves for selected models (VGG-19, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt-50).

The training durations and performance metrics of the deep learning models, retrained using the hyperparameters determined via Bayesian optimization, are presented in Table 7. In Table 7, Duration reports approximate wall-clock training time on Google Colab GPUs; inter-architecture differences should be interpreted as indicative of computational effort rather than as strictly comparable or benchmarked measurements. The column abbreviated as “Best val.” gives the peak validation accuracy achieved during tuning (a model-selection diagnostic only); accuracy, precision, recall, and F1-score refer to the held-out test split. The results highlight both the computational load and the classification performance. On the held-out test split, EfficientNetB3 achieved the highest classification accuracy (97.65%), followed by DenseNet121 (97.45%) and ViT (97.15%); AlexNet remained lowest at 83.89%. Peak validation accuracy during tuning reached 97.57% for ViT and 83.69% for AlexNet, but these values are reported only as tuning diagnostics because the same validation fold informed both Bayesian optimization and early stopping. Precision, recall, and F1-score values were reported alongside test accuracy for all models. EfficientNetB3 and DenseNet121 showed high values across all test metrics, with accuracies of 97.65% and 97.45%, respectively. AlexNet produced lower test results; however, its precision, recall, and F1-score values remained within a comparable relative range.

Figure 6 and Figure 7 present the confusion matrices for the retrained models on the test dataset (EfficientNetB3, ViT, XceptionNet, InceptionNet, and AlexNet in Figure 6; VGG-19, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt-50 in Figure 7), allowing class-wise prediction differences between architectures to be examined. EfficientNetB3 demonstrates generally balanced performance, particularly in terms of separating the human-induced damage class. The Vision Transformer (ViT) also shows stable behavior in most classes when the confusion matrix is examined. In contrast, XceptionNet displays increased misclassification in several classes, which may be related to its fewer optimized training epochs (Table 6). InceptionNet performs well in the chemical and crack classes, although some misclassifications are still observed.

Figure 6. Confusion matrices on the test set for EfficientNetB3, ViT, XceptionNet, InceptionNet, and AlexNet.

Figure 7. Confusion matrices on the test set for VGG-19, ShuffleNetV2, DenseNet121, MobileNetV2, and ResNeXt-50.

When examining the test data, it is observed that errors are more concentrated in certain classes in the AlexNet model. This situation is particularly evident in visually more complex damage types. The outputs obtained with the VGG-19 model showed a distribution without creating a significant imbalance between classes. The ShuffleNetV2 model produced stronger results in some classes and achieved relatively higher recognition success in human-caused damage examples.

EfficientNetB3 achieved the highest test-set accuracy among the evaluated models, while DenseNet121 maintained high consistency between classes. Despite its lightweight architecture, MobileNetV2 achieved high accuracy levels with its strong generalization ability. ResNeXt-50 utilized its expanded cardinality structure to increase its ability to distinguish between classes and achieved a high level of accuracy. In summary, the confusion matrices presented in Figure 6 and Figure 7 systematically highlight the impact of model complexity, number of parameters, and optimization processes on overall classification performance by revealing the class-specific strengths and weaknesses of different architectures.

The most common confusion appears between chemical decay and biological decay, where samples are often misclassified as one another, likely because of their similar visual characteristics such as discoloration and surface texture. Another notable ambiguity is observed between material loss and human-induced damage, suggesting that damage caused by external interventions can sometimes resemble natural deterioration. In addition, cracks are occasionally confused with material loss, as both involve visible structural disruptions such as fissures or missing parts. Although higher-performing models such as EfficientNetB3, ViT, and DenseNet121 reduce these errors to some extent, these class boundaries remain consistently challenging. Overall, this suggests that categories with overlapping visual features, especially those related to gradual degradation versus external impact, require more discriminative representations, pointing to a key area for future improvement.

On the test split, EfficientNetB3, DenseNet121, and ViT achieved the strongest overall performance (97.65%, 97.45%, and 97.15% accuracy, respectively). The higher test-set performance of ViT compared to relatively shallow CNNs such as AlexNet can be attributed to the self-attention mechanism, which effectively captures long-range dependencies between distant image patches. This capability is particularly beneficial for modeling extended crack patterns and widespread staining regions, which are less efficiently represented by convolutional layers with limited receptive fields unless significantly deepened. In addition, DenseNet’s feature reuse strategy and MobileNetV2’s use of depthwise separable convolutions also yielded strong test-set performance. These results suggest that both global context modeling and the ability to learn rich local feature hierarchies play an important role in this classification task.

Table 8 complements Figure 6 and Figure 7 by reporting per-class precision, recall, and F1-score for all ten architectures on the held-out test split, together with overall accuracy. The tabulated values confirm that EfficientNetB3, DenseNet121, and ViT sustain the strongest class-wise performance, whereas AlexNet remains weakest—particularly for chemical deterioration and undamaged surfaces—consistent with the confusion patterns discussed above.

Table 8. Per-class precision (P), recall (R), and F1-score (F1) (%) for all evaluated models on the test split.

Model	Biological			Chemical			Mechanical			Human Induced			Material Loss			Undamaged			Overall Acc. (%)
	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
EfficientNetB3	99.4	97.0	98.2	93.1	97.1	95.1	98.5	98.5	98.5	100.0	98.6	99.3	98.0	99.1	98.5	97.2	95.2	96.2	97.6
Vision Transformer (ViT)	99.1	95.8	97.4	93.8	96.8	95.3	94.2	99.1	96.6	99.1	99.4	99.2	98.8	98.0	98.4	98.2	93.2	95.6	97.2
XceptionNet	79.4	85.8	82.5	78.3	73.3	75.7	94.0	90.6	92.3	97.6	95.1	96.3	89.1	94.8	91.9	89.6	87.8	88.7	87.9
InceptionNet	87.6	85.8	86.7	80.1	80.3	80.2	92.1	95.8	93.9	99.1	94.5	96.7	94.9	92.2	93.5	86.9	92.5	89.6	90.1
AlexNet	96.5	81.3	88.3	70.2	80.6	75.0	84.8	84.5	84.6	88.3	90.8	89.5	91.4	82.9	86.9	76.5	83.0	79.6	83.9
VGG-19	93.5	89.3	91.4	76.5	93.3	84.1	96.3	87.3	91.6	97.7	97.1	97.4	90.3	91.9	91.1	91.5	81.0	85.9	90.2
ShuffleNetV2	98.5	95.8	97.1	93.4	95.1	94.2	98.1	96.4	97.2	99.1	98.0	98.5	97.4	98.0	97.7	93.1	96.6	94.8	96.6
DenseNet121	98.8	96.1	97.4	94.5	94.2	94.3	99.1	98.5	98.8	99.4	99.7	99.5	98.0	99.1	98.5	94.7	96.9	95.8	97.4
MobileNetV2	97.9	97.3	97.6	93.7	95.4	94.5	97.6	97.6	97.6	98.8	98.6	98.7	98.8	96.2	97.5	93.7	95.6	94.6	96.8
ResNeXt-50	97.6	95.3	96.4	82.5	95.9	88.7	97.9	97.0	97.4	100.0	91.7	95.7	97.3	93.6	95.4	95.9	94.9	95.4	94.7

Color legend:      ≥95%         85–95%      $<85\%$.

7. Discussion and Practical Implications

The present study sits at the intersection of architectural conservation, building pathology, and artificial intelligence. Background Section 2, Section 3 and Section 4 introduce deterioration mechanisms and conventional assessment practices for readers outside computer vision, whereas dataset construction, model training, and quantitative evaluation are centralized in Section 5.1 and Section 6. The discussion below emphasizes interpretation of the test-set results, practical implications, and limitations.

A critical observation in the confusion matrices is the recurring ambiguity between chemical and biological decay. From a building physics perspective, these deterioration types often share similar visual signatures, such as surface discoloration and texture alteration. Similarly, the confusion between material loss and cracks highlights that structural disruptions often manifest as a continuum rather than isolated events. These “borderline cases” suggest that while AI can significantly narrow down the diagnosis, the final verdict, especially for interventions on heritage buildings, still benefits from expert tactile inspection and monitoring regulations.

The corpus reflects the geographic and material diversity described in Section 5.1, supporting evaluation under realistic outdoor acquisition conditions.

The stratified 70/20/10 partition (Table 3) provided a computationally efficient baseline for comparing ten architectures under identical conditions. While the current single-split approach demonstrates high internal consistency, incorporating k-fold cross-validation in future iterations of this research could further enhance the statistical confidence, particularly as more diverse datasets are integrated. Such an approach would build upon the extensive comparative framework established in this paper.

A limitation of the present study is the absence of an external validation dataset. Although the corpus includes images collected from different countries, material types, environmental conditions, and acquisition settings (Section 5.1), the reported performance reflects generalization within the distribution represented by the current dataset rather than transfer to an independently curated heritage corpus. Caution should therefore be exercised when extrapolating these findings to historical buildings characterized by substantially different architectural traditions, climatic conditions, or conservation histories. Future studies incorporating external datasets, additional geographic regions, and cross-domain evaluation protocols would enable a more rigorous assessment of model robustness and transferability.

Experiments were executed in the Google Colab Pro+ environment using NVIDIA Tesla T4 and NVIDIA A100 GPUs (NVIDIA Corporation, Santa Clara, CA, USA) with approximately 12–25 GB of available RAM depending on the session. The wall-clock training durations in Table 7 (roughly 10–18 h in the present runs) should be interpreted as approximate indicators of computational effort under these practical conditions, not as standardized or reproducible hardware benchmarks. On the same infrastructure, single-image inference required about 10–50 ms per forward pass, with the exact value depending on architecture complexity; practitioners should compare latency, parameter count, and memory when selecting a deployment model. For heritage monitoring, lightweight architectures offer reduced memory footprints for edge-device integration, while models like EfficientNetB3, ViT, or DenseNet121 prioritize accuracy for detailed offline analysis.

The proposed framework is not intended to replace conservation experts or comprehensive Structural Health Monitoring (SHM) systems. Rather, it should be regarded as an intelligent visual decision-support tool that supports routine inspections through the rapid and standardized identification of deterioration patterns from images acquired during field surveys, UAV campaigns, mobile devices, or fixed monitoring platforms. When combined with expert evaluation, non-destructive testing, historical documentation, and sensor-based SHM [38,39], image-based damage classification can provide complementary information for long-term condition assessment. Periodic image acquisition may also facilitate tracking of deterioration progression, supporting preventive conservation, maintenance prioritization, and the scheduling of restoration interventions. Integrating AI-assisted visual inspection with existing SHM and heritage conservation workflows may therefore enhance the efficiency and continuity of heritage monitoring practices.

From a managerial perspective, the proposed image-only workflow is compatible with national-scale heritage inventories: field teams can upload façade photographs to a centralized repository, and a selected model (e.g., MobileNetV2 for edge deployment or EfficientNetB3 for archival review) can pre-sort deterioration types before expert sign-off. Doubling the dataset would approximately double GPU training time and storage requirements under the same BO protocol, but inference latency would remain unchanged. Basic operation of the tools—dataset curation, transfer-learning fine-tuning, and confusion-matrix review—could be integrated into architecture and conservation studio courses; advanced hyperparameter tuning and model governance would nonetheless benefit from post-graduate specialization in computational heritage science.

Resizing all inputs to 224 × 224 pixels remains a methodological limitation for extremely fine cracks and localized efflorescence. Quantitative evaluation of input resolution would provide stronger evidence; higher-resolution inputs (e.g., 384 × 384 and 512 × 512), patch-based analysis, and multi-scale learning strategies are recommended in future benchmark extensions.

Finally, automated damage classification is advisory; false negatives on safety-critical defects could delay interventions, and dataset bias may under-represent certain materials or climates. Because prior restoration or retrofit status was not distinguished or recorded during acquisition (Section 5.1), generalization with respect to intervention history cannot be assessed from the present corpus. Similarly, because splits were not constrained by building identity (Section 5.1), reported test-set metrics reflect image-level generalization within the acquisition pool rather than strict structure-wise holdout; caution is warranted when extrapolating to unseen buildings or façades. Outputs should therefore be combined with expert inspection and monitoring regulations applicable to heritage structures.

8. Conclusions

Using the publicly released heritage image corpus (Section 5.1; [40]), ten deep learning architectures were fine-tuned through transfer learning. Near-balanced class distributions were maintained (3400 images per damage class and 3000 undamaged samples), and the mild undersampling of the undamaged class did not materially reduce its test-set recall in the best-performing models (Figure 6 and Figure 7; Table 8). The proposed framework addresses crack-dominated mechanical deterioration rather than the full spectrum of structural failures (e.g., crushing, buckling, or collapse); results should therefore be interpreted within the scope of visually observable crack-related deterioration. Each model was evaluated separately. On the independent test split, EfficientNetB3 achieved the highest accuracy (97.65%), followed by DenseNet121 (97.45%) and ViT (97.15%). ViT and MobileNetV2 recorded the highest validation accuracies during tuning (97.57% and 97.07%, respectively), but validation scores were used only for model selection; final comparisons and conclusions rely on test-set metrics. These findings show that the dataset is suitable for distinguishing different damage types in historical structures. It can also be applied across several deep learning models.

The reported accuracy values are generally consistent with those presented in previous studies on cultural heritage damage detection, which used various detection methods and classification approaches. For instance, Maras and Sariyildiz [18] achieved 95.1% accuracy in a three-class post-earthquake building damage detection task using UAV images and Mask R-CNN, while Hatir et al. [22] reported 99.4% accuracy for an eight-class problem focused on material defects in stone monuments. Direct comparisons are limited, however, due to differences in task definitions, number of classes, data balance, and imaging conditions. Notably, the dataset used in this study consists of six classes encompassing a wide variety of construction materials, which adds complexity and better reflects real-world conditions. The relatively high accuracy achieved here can be attributed to this near-balanced and diverse dataset, as well as the consistent training and optimization protocol applied across all models.

Future work may extend the scope of the dataset by including additional building typologies and observed damage conditions. At the national scale, the workflow supports progressive assembly of a representative building-surface database from distributed field campaigns. Expanding the corpus would increase storage and retraining cost roughly in proportion to sample size, while leaving single-image inference time unchanged; practitioners could therefore prioritize incremental dataset growth coupled with lightweight deployment models. Training conservation professionals to operate the pipeline is feasible within existing architecture curricula for image labelling and inference review, whereas Bayesian optimization and model maintenance are better suited to specialized postgraduate programs. Future studies are encouraged to incorporate building-level annotations and adopt structure-wise data partitioning strategies to enable a more reliable evaluation of the generalization capability of deep learning models for historical building damage classification. Cross-dataset evaluation on independently annotated heritage corpora from additional geographic regions and previously underrepresented typologies would further support assessment of model robustness and transferability beyond the present acquisition pool. Future research may also extend the proposed framework by integrating image-based classification with sensor measurements, environmental data, material characterization, and digital twin technologies [38,39] to establish more comprehensive adaptive monitoring systems for cultural heritage assets. Employing high-resolution spatial documentation techniques, such as 3D scanning and drone-assisted surveys, could improve the understanding of long-term structural change. Within this framework, the present study contributes a reliable reference point for the development of data-oriented AI methods in heritage conservation and restoration research.