Intelligent Defect Recognition of Glazed Components in Ancient Buildings Based on Binocular Vision

Abstract

Glazed components in ancient Chinese architecture hold profound historical and cultural value. However, over time, environmental erosion, physical impacts, and human disturbances gradually lead to various forms of damage, severely impacting the durability and stability of the buildings. Therefore, preventive protection of glazed components is crucial. The key to preventive protection lies in the early detection and repair of damage, thereby extending the component’s service life and preventing significant structural damage. To address this challenge, this study proposes a Restoration-Scale Identification (RSI) method that integrates depth information. By combining RGB-D images acquired from a depth camera with intrinsic camera parameters, and embedding a Convolutional Block Attention Module (CBAM) into the backbone network, the method dynamically enhances critical feature regions. It then employs a scale restoration strategy to accurately identify damage areas and recover the physical dimensions of glazed components from a global perspective. In addition, we constructed a dedicated semantic segmentation dataset for glazed tile damage, focusing on cracks and spalling. Both qualitative and quantitative evaluation results demonstrate that, compared with various high-performance semantic segmentation methods, our approach significantly improves the accuracy and robustness of damage detection in glazed components. The achieved accuracy deviates by only ±10 mm from high-precision laser scanning, a level of precision that is essential for reliably identifying and assessing subtle damages in complex glazed architectural elements. By integrating depth information, real scale information can be effectively obtained during the intelligent recognition process, thereby efficiently and accurately identifying the type of damage and size information of glazed components, and realizing the conversion from two-dimensional (2D) pixel coordinates to local three-dimensional (3D) coordinates, providing a scientific basis for the protection and restoration of ancient buildings, and ensuring the long-term stability of cultural heritage and the inheritance of historical value.

1. Introduction

Glazed components have been utilized in ancient Chinese architecture for millennia, embodying significant historical and cultural value [1]. However, over time, glazed components progressively exhibit various forms of damage due to environmental erosion, physical impacts, and human interference [2]. The combined effects of factors such as temperature fluctuations [3], thermal expansion [4], and structural vibrations [5] can induce thermal expansion cracks or stress fractures in glazed components [6]. These cracks not only substantially weaken the structural integrity of the glazed components [7], but also ultimately lead to their rupture, severely compromising the durability and stability of the building [8]. Simultaneously, prolonged exposure to climate change and environmental factors [9] often results in the spalling of the surface of glazed components [10]. The spalling of the surface not only strips the glazed components of their original luster and decorative effect [11] but also diminishes the artistic expression and historical significance of the building, hindering the presentation of its original historical characteristics. Furthermore, as brittle materials, glazed components are susceptible to stress concentration under external forces, resulting in localized damage [12,13,14,15]. Particularly in areas subjected to heavy loads, glazed components may experience localized fractures or shattering due to uneven external forces or improper repair techniques [16], further compromising the overall structural stability of the building. This degradation of mechanical properties not only jeopardizes the physical safety of the building, but also threatens its historical authenticity and cultural significance [17,18]. In recent years, the field of international cultural heritage conservation has increasingly embraced digital technologies for refined monitoring and analysis. A series of studies, particularly in Europe, have widely adopted approaches such as Historic Building Information Modeling (HBIM) [19], multi-source photogrammetry [20], and multimodal imaging techniques—including infrared thermography and hyperspectral scanning [21] for the detection and long-term monitoring of surface damage in historic structures. By integrating information across different scales and physical properties, these methods have substantially improved the precision of damage identification and spatial localization, thereby enabling more systematic and sustained preservation strategies for historical heritage.

Despite the significant progress made in recent years in image-based computer vision approaches for damage identification and detection [22,23,24,25], particularly in achieving high-precision classification and localization through semantic segmentation and object detection techniques [26,27,28,29,30], most existing methods rely heavily on two-dimensional RGB images for feature extraction and pattern recognition [31,32,33]. This reliance introduces several critical limitations. Traditional RGB images only contain color and texture information, lacking true depth and spatial positioning attributes [34,35,36,37], which prevents the mapping of identified damages into real-world physical space. As a result, it becomes difficult to accurately quantify the actual width, length, and depth of damage features [38,39,40], thereby compromising the scientific validity and comparability of structural damage assessments [41]. Moreover, factors such as camera viewing angle, focal length, and imaging distance during image acquisition directly influence the pixel dimensions of targets in images, leading to scale inconsistencies across varying conditions [42]. This can result in false positives and missed detections in severe cases. Damage recognition models based solely on 2D imagery are inherently incapable of perceiving the geometric relationships between damages and structural surfaces. Consequently, they struggle to distinguish between superficial and structural penetrating damages and cannot support spatial tracking or modeling of damage paths. In real-world engineering contexts, the physical dimensions of damage are essential indicators for determining the degree of structural performance degradation, evaluating repair strategies, estimating restoration efforts, and formulating preventive conservation measures [43,44,45,46,47,48]. Image-only models lack the quantitative capabilities required for such tasks [49], which severely limits their practical value in high-precision damage detection scenarios [50,51,52]. Therefore, there is an urgent need to establish a methodological framework that integrates visual recognition with spatial reconstruction capabilities—one that combines image-based damage identification with depth maps, camera parameters, or point cloud data to recover the true three-dimensional scale of defects within physical space.

The contributions of this study are as follows:

• Development of a Multi-Type Damage Dataset for Glazed Architectural Components. To address the diverse morphological characteristics and significant scale variations of surface damages, cracks and spalling on glazed architectural components, this study establishes a specialized dataset focusing on ancient glazed structures. In particular, it incorporates a full processing pipeline for detecting and analyzing the prevalent crack and spalling damages found on the Nine-Dragon Wall. This dataset provides a foundational resource for the digital monitoring and quantitative scale assessment of cultural heritage components.
• Proposal of a Deep Learning-Based Detection Algorithm for Glazed Surface Damage with Complex Textures. This study designs a deep neural network architecture tailored for detecting damages in the intricate textures of glazed surfaces. The CBAM is integrated into the backbone network and applied to the output of each feature processing stage, enabling the model to learn highly discriminative and semantically rich features at early stages of extraction. This attention-enhanced architecture significantly improves feature representation capabilities and provides more accurate and robust semantic support for downstream damage detection tasks, ultimately achieving higher precision in image-based defect recognition.
• Construction of a Depth Estimation and Scale Restoration Fusion Algorithm for Accurate 2D to 3D Mapping. Based on the depth information obtained from the detected damage regions, this study introduces a 3D coordinate back-projection method using pre-calibrated intrinsic camera parameters to transform the 2D pixel-based segmentation results into real-world physical space. This approach enhances the geometric accuracy of damage quantification and improves the spatial interpretability of the detection outcomes. It provides reliable and quantifiable 3D data support for subsequent tasks such as structural health analysis, restoration planning, and long-term monitoring of architectural heritage components.

2. Related Work

2.1. Traditional Methods

In the detection of structural deterioration in glazed components and similar heritage elements, traditional methods predominantly rely on manual inspection and contact-based measurements [53,54]. Such inspections are typically conducted by experienced conservation professionals using visual observation, manual measurement, and percussion-based auditory techniques to identify surface damages such as cracks and spalling. These methods are characterized by their simplicity and intuitive operation; however, the reliability of the results is highly contingent upon the inspector’s expertise and is often influenced by subjective judgment and environmental variability. This undermines the consistency and comparability required for long-term monitoring. Internationally, certain cultural heritage institutions have integrated auxiliary instruments—such as high-precision vernier calipers, crack width gauges, and optical microscopes [55,56,57]—into traditional inspection workflows to enhance measurement accuracy. Contact displacement sensors and handheld microscopic probes have also been employed. Nevertheless, these approaches face substantial limitations when applied to large-scale surfaces, structurally intricate components, or elevated positions. Challenges include low efficiency, high labor intensity, and the risk of inflicting secondary damage to delicate heritage materials. In China, architectural heritage conservation practices [58,59,60] similarly emphasize manual inspection supplemented by localized measurement. However, in the face of increasing demands for high-precision, rapid, and non-contact diagnostic techniques, the limitations of conventional methods are becoming increasingly pronounced.

2.2. Deep Learning Methods

With the rapid advancement of computer vision and artificial intelligence, deep learning–based image recognition techniques have been increasingly adopted in the detection of damage in cultural heritage components. Both domestic and international studies have employed convolutional neural networks (CNNs) [61], object detection models such as Faster R-CNN and the YOLO series [62,63], and semantic segmentation frameworks including U-Net, DeepLab, and the Mask R-CNN family [64,65,66], to enable automated identification and localization of defects such as cracks and spalling.

In the international context, earlier research primarily focused on structural damage in civil infrastructure, such as concrete and steel structures. For instance, some scholars combined Faster R-CNN with image enhancement strategies to improve crack detection performance under varying illumination conditions [67]. Others utilized multi-scale convolutional features and attention mechanisms to enhance the extraction of subtle damage patterns [68]. In China, research efforts have increasingly integrated practical needs from both civil engineering and cultural heritage conservation [69,70], leading to the development of lightweight and high-robustness detection models. Examples include the integration of YOLOv8 with feature pyramid networks (FPN) to improve the detection of small-scale damage features [71], as well as the incorporation of Transformer architectures or multi-modal data fusion approaches [72] to enable more comprehensive, data-driven damage recognition and condition assessment. However, existing deep learning approaches still face two major challenges when applied to heritage components. First, most models are based on monocular RGB imagery, lacking the ability to recover the true spatial dimensions of detected defects. Second, their adaptability remains limited when confronted with complex surface textures, lighting variations, and the coexistence of multiple types of damage.

3. Method

3.1. Technical Route for Automatic Damage Identification Method and Scale Restoration of Glazed Components

An automated processing flow integrating image recognition, depth perception, and scale restoration is developed for various types of damage features on the surface of glazed components, such as cracks, spalling, and missing parts. The process mainly involves the following steps: (1) binocular vision RGB-D image acquisition based on frame sequences; (2) semantic segmentation and instance recognition of damaged areas; (3) a fusion algorithm for depth estimation and scale restoration; and (4) 3D coordinate recovery based on internal and external camera parameters. By designing an end-to-end neural network structure, a precise mapping from 2D images to real-scale 3D space is achieved, ensuring geometric accuracy and spatial interpretability in damage detection. The technical roadmap is presented in Figure 1.

3.2. Binocular Vision System Data Acquisition and Enhancement

In this study, a calibrated binocular stereo vision system was employed to capture surface image data of glazed components from historic architecture. The stereo camera setup was calibrated by configuring a fixed baseline distance B and known focal length f, with parallel optical axes between the two cameras to minimize baseline error. An Intel RealSense D455f depth camera was utilized, the manufacturer of the Intel RealSense D455f is Intel. The company’s headquarters are located in Santa Clara, California, in the United States. The image synchronization was achieved through the RealSense SDK to ensure temporal alignment between RGB and depth images. The image acquisition was performed at a resolution of 1280 × 720 pixels and a frame rate of 30 fps. During data acquisition, intrinsic camera parameters were simultaneously extracted. These parameters, which include focal lengths, principal point coordinates, and distortion coefficients, describe the imaging model of the camera and are essential for establishing geometric correspondences between image coordinates and 3D spatial coordinates. These intrinsic parameters provided the foundational input for subsequent image registration and scale recovery processes.

The image acquisition was conducted at the glazed Nine-Dragon Wall in Beihai Park, Beijing, a representative example of traditional glazed architectural components. This structure, characterized by outdoor glazed decorative surfaces, is particularly prone to damage such as cracking and spalling at the lower sections due to stress concentration. Image pairs were captured from multiple reasonable viewpoints and distances within the visible range of the glazed wall to ensure spatial diversity and enhance the generalization capability of the model in real-world scenarios. The collected dataset consists of time-stamped synchronized RGB-D image pairs and the corresponding intrinsic calibration matrices, serving as the data foundation for image registration, scale restoration, and 3D damage recognition tasks. Furthermore, to improve recognition robustness in low-texture regions, structured light projection was incorporated during acquisition to enhance the visibility of surface texture features. A schematic diagram of the acquired data is shown in Figure 2.

Figure 3 presents a point cloud model of the entire Nine Dragon Wall, scanned and reconstructed using a FARO Premium scanner. This point cloud accurately represents the surface geometry and detailed information of the component, with its density and distribution effectively illustrating the component’s complex structure. Using image registration and point cloud fusion technologies, image data from different perspectives were successfully converted into a high-precision 3D point cloud model, laying the groundwork for subsequent analysis and processing.

3.3. Scale Uncertainty Analysis

Because image processing is inherently pixel-based, missing scale information is a key issue affecting geometric reconstruction accuracy. This paper solves the scale uncertainty problem by restoring the scale method. Baseline error in a binocular system significantly impacts depth estimation. The depth estimation and coordinate transformation formula for a binocular system is based on parallax d = xl − xr, where xl and xr are the pixel values of the left and right images at the same timestamp. The depth value is calculated as follows:

$$Z={\displaystyle \frac{f\cdot B}{d}}$$

(1)

Subsequently, each pixel coordinate (u, v) in the depth image was back-projected into real-world three-dimensional coordinates (X, Y, Z) by integrating the depth image with the intrinsic parameters of the calibrated camera. This inverse projection follows the pinhole camera model, enabling accurate mapping from the 2D image plane to the 3D physical space. The transformation is described by the following equations:

$$\left\{\begin{array}{l}X={\displaystyle \frac{(u-c_x)\cdot Z}{f_{\mathrm{x}}}}\\ Y={\displaystyle \frac{(v-c_y)\cdot Z}{f_y}}\end{array}\right.$$

(2)

where (c_x, c_y) are the coordinates of the principal point in the image, and f_x, f_y are the focal lengths.

The core principle for calculating the real-world size and distance of a lesion is to convert the pixel size in the 2D image into physical dimensions in 3D space using the camera’s intrinsic parameters and the object’s depth information. The dynamically scaled focal length (f_x, f_y) is dynamically adjusted based on the ratio of the current image resolution to the native resolution. Calculating the pixel-to-meter conversion factor. With the average depth and the adjusted focal length, a crucial conversion factor can be calculated: the real-world size per pixel. This factor represents the physical distance represented by a pixel at the lesion’s depth. The formula for this conversion factor is as follows:

$$\mathit{meters}\_per\_pixel={\displaystyle \frac{Z_{avg}}{f_{avg}}}$$

(3)

where Z_avg is the average depth of the damage, and f_avg is the average focal length after dynamic scaling. To calculate the true size, with the meters_per_pixel conversion factor, simply multiply the pixel measurement by this factor to obtain the true size in meters. Pixel-level size measurement first extracts the exact pixel outline of the damage, then uses the minimum bounding rectangle to obtain the length pixels and width pixels of the damage, which are then converted to true size using the following formula:

$$\begin{array}{c}length\_meter=length\_pixels\times meters\_per\_pixel\hfill \\ \mathit{width}\_meter=\mathit{width}\_pixels\times meters\_per\_pixel\hfill \end{array}$$

(4)

In addition to the size, the coordinates of the center point of the damage in 3D space (X, Y, Z) are also calculated. The pixel center point of the damage on the image (center_x_px, center_y_px) is calculated using the image moment. Then, the real-world three-dimensional coordinates were computed using geometrically derived back-projection formulas, by integrating the pixel centroid, average depth, and the intrinsic parameters of the camera.

$$\begin{array}{l}\mathrm{X}=(center\_x\_px-c_x)\cdot {\displaystyle \frac{Z_{avg}}{f_x}}\\ Y=(center\_y\_px-c_y)\cdot {\displaystyle \frac{Z_{avg}}{f_y}}\\ Z=Z_{avg}\end{array}$$

(5)

By obtaining a pixel-level segmentation mask, using the depth map to obtain the average depth, and combining it with camera intrinsic parameters, the pixel size and position are accurately converted to real-world physical dimensions and 3D coordinates.

3.4. Design of Scale Restoration Algorithm

Existing object recognition algorithms treat all feature channels and spatial locations equally during feature extraction, without prioritizing important information and regions. To effectively enhance the representational capacity of critical regions while maintaining computational efficiency, the CBAM was adopted. CBAM applies attention mechanisms sequentially along the channel and spatial dimensions of feature maps, enabling dynamic weighting of salient features across multiple levels. Compared with other attention mechanisms, CBAM offers a lightweight structure with low computational overhead and demonstrates superior performance in improving model accuracy, making it particularly suitable for fine-grained damage recognition tasks. In this study, CBAM are integrated into the backbone network and applied to the output of each C2f module. This integration allows the network to learn more discriminative features at the early stages of feature extraction, thereby providing more accurate and semantically meaningful representations for subsequent detection and segmentation tasks.

To achieve damage recognition from images to real-world spatial scales, this study designed a deep neural network architecture incorporating a scale restoration module. The overall framework consists of the following modules: We propose the Restore Scale Identification (RSI) object detection method, which can be divided into three parts: a backbone feature extraction module, a neck feature fusion module, and a head detection output module. First, the input image undergoes multiple layers of convolution (Conv) followed by a feature extraction module (C2f) to gradually extract multi-scale semantic features. An attention mechanism is introduced at the end to enhance the recognition of fine-grained features, such as damage. The neck layer employs a multi-layer upsampling and feature concatenation strategy to fuse feature information from different levels, thereby enhancing small object detection capabilities. Finally, a multi-scale detection branch, Detect, is placed in the Head part to output the bounding box (Bbox), category (Cls), and loss function supporting multi-target, high-precision damage identification tasks.

Figure 4 shows the internal structure of the CBAM, which enables neural networks to dynamically focus on important features. CBAM consists of two serially connected submodules: a Channel Attention Module (CAM) and a Spatial Attention Module (SAM). The CAM inputs a feature map F of size H × W × C. After processing by the CBAM, an enhanced feature map F’’ of size H × W × C is generated. The CAM then performs global max pooling and global average pooling on the feature map F, generating two feature vectors of size 1 × 1 × C. These two feature vectors share a multi-layer perceptron (MLP). The MLP consists of a dimensionality reduction layer, a ReLU activation function, and a dimensionality increase layer. The outputs of the MLP are element-wise summed and then passed through a sigmoid activation function to generate a channel attention weight vector M_c of size 1 × 1 × C. The output is a channel-weighted feature map F’, calculated as follows:

$$F\prime =M_c\otimes F$$

(6)

The spatial attention module (SAM) inputs the channel-weighted feature map F′, performs maximum pooling and average pooling on F′ in the channel dimension, and obtains two feature maps of size H × W × 1. These two feature maps are concatenated to obtain a feature map of size H × W × 2. A standard convolution layer is applied to this concatenated feature map. Finally, after passing the Sigmoid activation function, a spatial attention weight map M_s of size H × W × 1 is obtained. The final enhanced feature map F″ is output. The calculation method is:

$$F\prime \prime =M_s\otimes F\prime$$

(7)

4. Result and Discussion

4.1. Dataset Creation

The depth camera system is employed to acquire RGB images from the left camera as the primary input for damage detection, with depth images temporally synchronized via timestamp alignment. Both intrinsic and extrinsic camera parameters are recorded to facilitate subsequent 3D coordinate reconstruction. A custom dataset, named the Semantic Segmentation Dataset of Glazed Tile Damage Images (sS-DGDID), was constructed by manually annotating regions of cracks, spalling, and material loss using the Labelme v5.1.0, with semantic polygon labels assigned as liefeng (crack) and tuoluo (spalling). Background regions were also labeled to assist the model in distinguishing between damaged and undamaged areas. In total, 3472 usable RGB images were collected, covering the main structure of the Nine-Dragon Wall and surrounding glazed architectural elements. These images were captured from diverse and strategically chosen viewpoints and distances to ensure comprehensive spatial coverage of component distribution and damage states, thereby enhancing the spatial generalization ability of the model. The dataset was partitioned into 70% for training, 20% for validation, and 10% for testing, as illustrated in Figure 5.

4.2. Dataset Train

The RSI architecture was designed by incorporating both attention mechanisms and a scale restoration branch. The training experiments were conducted on a platform equipped with an Intel Core i9-13900KF CPU, NVIDIA RTX 4090 desktop GPU (24 GB VRAM), running Ubuntu 22.04 LTS with Python 3.10, PyTorch 2.0, and CUDA 11.8. Taking into account both model accuracy and GPU memory constraints, the optimal configuration was determined through iterative training and hyperparameter tuning. The input image size was set to 640 × 640 pixels, and the batch size was fixed at 16. To mitigate overfitting and improve convergence stability, the Adam optimizer was employed with an initial learning rate of 0.001 and a learning rate decay factor of 0.75. The total number of training epochs was set to 300. Loss functions were defined as a weighted combination of classification loss (BCE Loss) evaluated using Precision and Recall, bounding box regression loss (CIoU Loss) assessed by IoU, and segmentation loss (Dice Loss) evaluated using F1 score, IoU, and Recall. The training process required approximately 12 h, and the average inference time per image was around 40 ms. The detailed hyperparameter settings are summarized in

Table 1. Model training parameters.

Size	Batch Size	Optimizer	Number of Epoch	Initial Learning Rate	Learning Rate Decay
640	16	Adam	300	0.001	0.75

.

The intersection over union (IOU) is used to measure the degree of overlap between the predicted area and the true area:

$$IOU={\displaystyle \frac{TP}{TP+FP+FN}}$$

(8)

TP: The number of pixels predicted to be positive and actually positive (True Positive); FP: The number of pixels predicted to be positive but actually negative (False Positive); FN: The number of pixels predicted to be negative but actually positive (False Negative).

Precision is used to measure how many of the samples predicted to be positive are actually positive samples:

$$\mathit{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

(9)

Recall is used to measure how many of the truly positive samples are correctly predicted as positive:

$$\mathit{Recall}={\displaystyle \frac{TP}{TP+FN}}$$

(10)

The comprehensive evaluation F1-score combines Precision and Recall, and the formula is as follows:

$$F1={\displaystyle \frac{2\times \mathit{Precision}\times \mathit{Recall}}{\mathit{Precision}+\mathit{Recall}}}$$

(11)

It is the harmonic mean of Precision and Recall. The closer F1 is to 1, the better the model finds the majority of positive examples and ensures that the majority of predicted positive examples are correct. If Precision is high but Recall is low, or if Recall is high but Precision is low, the F1-score will be significantly lower. In tasks with an imbalanced sample class, the F1-score better reflects the model’s true performance than Precision or Recall alone.

4.3. Damage Identification Results and Discussion

To verify the effectiveness of the proposed method for damage identification and scale restoration of glazed components, this section conducts experimental analysis based on the collected binocular RGB-D data. The detection results for multiple types of damage, including cracks and spalling, are visualized and evaluated. First, semantic segmentation results from different test samples are compared to analyze the network’s performance under complex texture backgrounds and its ability to preserve contour integrity. Subsequently, using the 3D size calculated from depth information, the scale restoration accuracy of the detection results is quantitatively validated. As shown in

Table 2. Comparison results of various methods in terms of accuracy, recall rate, and F1-score. The bold ones are the ones with the highest recognition accuracy for each method in each category.

	Class	IoU	P	R	F1
U-Net	tuoluo	74.1	88.2	84.1	85.5
	liefengg	70.5	82.4	80.4	81.4
DeepLabv3+	tuoluo	77.2	87.0	83.8	85.4
	liefengg	72.7	85.3	78.8	82.2
YOLOv8-Seg	tuoluo	80.6	93.5	90.4	92.4
	liefengg	74.9	94.1	89.6	93.9
Ours	tuoluo	92.2	97.1	94.1	96.5
	liefengg	86.4	95.7	95.9	95.5

, the accuracy of the proposed method improves by 19.1%, 16%, and 8.7% respectively, compared with the other three methods. As can be seen from the figure, compared with YOLOv8-Seg in Figure 6a, the upper area is incorrectly identified as blue-colored spalling. In the middle area, the green-colored cracks are misclassified as blue-colored spalling, and in the lower part, multiple damage areas are not detected at all. In Figure 6b, the green cracks in the middle region are not fully recognized by YOLOv8-Seg; some parts are missed, and the upper part of the image shows misclassification of undamaged areas as cracks. In Figure 6c, both the upper and middle regions exhibit misclassifications, such as spalling areas being mistaken for green cracks in the middle area. In Figure 6d, misclassifications and missed detections are also observed. Compared with the proposed method, DeepLabv3+ also shows incorrect and missed detections: In Figure 6a, there are both misclassified regions and completely undetected damage areas. In Figure 6b, the number of correctly recognized areas is significantly fewer than that achieved by the proposed method. In Figure 6c, there is misclassification in the middle area. In Figure 6d, the results exhibit false positives. Compared with the proposed method, U-Net demonstrates even more areas with incorrect classification and failure to detect damage.

Although these three methods show relatively good recognition performance in some areas, they still suffer from false and missed detections. These errors are mainly due to the inadequate handling of local features under complex textures by YOLOv8-Seg, the atrous convolution module DeepLabv3+, and the traditional encoder-decoder-based U-Net. This is particularly evident in locating tiny damages with complex morphology and blurry edges, which are difficult to identify. In contrast, the proposed method incorporates CBAM, which enhances attention to damage areas under complex textures and backgrounds by dynamically weighting important regions. This significantly mitigates issues of false detection and missed detection, thus validating the advantage of the proposed method in fine-grained damage recognition. The experimental validation demonstrates that, after the introduction of the attention mechanism, the proposed method achieves more accurate identification of the two damage categories and reduces the probability of misclassification. Results are shown in Table 2, and the visualized outcomes are presented in Figure 6.

4.4. Scale Restoration Accuracy Evaluation

For data collection on the Nine Dragon Wall, a ground-based 3D LiDAR. The FARO Premium is manufactured by FARO Technologies, Inc., which is headquartered in Lake Mary, Florida, USA. FARO Premium was used for high-precision data collection. The scanner settings were: color scanning, 10 min per station, and a total of 16 stations. After the scanning stations were stitched together, the maximum error of the 3D laser point cloud was within 4 mm, the overall stitching accuracy was 1.4 mm, and the point cloud density was 800,000 points/m². The damage to each part was measured and visualized as shown in Figure 7.

The actual size results identified by the depth camera are compared with the measured FARO 3D LiDAR point cloud, which is used as the true value of the scale for comparison. Several damages are compared, as shown in Figure 8.

After analyzing the predicted results against the ground truth acquired from 3D LiDAR point cloud measurements, it was found that the discrepancy between the two remained within ±10 mm. A deeper analysis revealed that the residual error primarily originated from the following sources: First, the inherent accuracy limitations of the depth camera may introduce slight deviations in local regions. Second, during the image segmentation process, edge effects and the smoothing of damage boundaries could lead to inaccuracies in delineating the extent of damage. Third, variations in material reflectance may affect the precision of depth acquisition, thereby introducing additional measurement errors. Despite these factors, the proposed method is still capable of effectively predicting the 3D information of surface damages. The trained model successfully generates high-quality, pixel-level damage masks, providing accurate inputs for subsequent geometric computations. Through the implementation of preprocessing techniques such as median filtering and smoothing, noise and outliers in the depth images were effectively suppressed, ensuring the reliability of average depth estimation. Additionally, the scale restoration algorithm dynamically adjusts the intrinsic parameters of the camera based on image resolution, mitigating measurement errors caused by image resizing and enhancing the robustness of the calculations.

An error level within ±10 mm is considered relatively high accuracy in conventional visual scale estimation methods, and it holds practical significance for the restoration and conservation of glazed components in architectural heritage. Damage in glazed tiles typically manifests as fine cracks or micro-spalling localized to small regions. Within this ±10 mm margin, the method can effectively support preliminary repair planning, component size estimation, and reconstruction of missing areas, particularly in scenarios where high-precision laser scanning is not feasible. The computational results are summarized in

Table 3. Damage prediction value, 3D laser point cloud measurement value and their difference.

	(a) (Length × Width)/m		(b)		(c)
Predicted value	0.289	0.112	0.175	0.123	0.224	0.127
Measured value	0.292	0.113	0.181	0.133	0.221	0.124
Difference	0.003	0.001	0.006	0.010	−0.003	−0.003

.

By adding camera intrinsics between the depth image and the RGB image, Figure 4 shows the transformation from pixel coordinates to local 3D coordinates. This process, which transforms 2D pixel coordinates into 3D local coordinates, is achieved by fusing the depth image with the RGB image and utilizing camera intrinsics. This process is a core step in computer vision and 3D reconstruction and is the foundation for damage size measurement. Pixel coordinates: These are the 2D coordinates of the damage on the image obtained from the RGB image using the segmentation model. Depth information: The grayscale value in the depth image represents the distance of the pixel along the Z axis in the camera coordinate system. By combining pixel coordinates and depth information with camera intrinsics, we can back-project the 2D coordinates back into 3D space. This transformation enables us to extract the true geometric information of an object from a 2D image that originally lacks depth information, as shown in Figure 9.

5. Conclusions

This study effectively integrates the perception capability of deep learning with the camera model-based back-projection mechanism in computer vision. By incorporating depth information, the proposed method extends traditional 2D image-level damage detection to the recognition of real-world dimensions and 3D spatial locations. Utilizing RGB-D data and a scale recovery algorithm, our approach enables accurate identification and measurement of damages in glazed architectural components without relying on high-precision laser scanners, demonstrating strong practicality and applicability in real-world heritage conservation tasks. The dataset employed in this study primarily consists of glazed components, which, while representative, still have limitations in terms of material diversity, environmental conditions, and structural forms. To generalize the applicability of the method to other types of cultural heritage materials—such as brick masonry, timber structures, and stone carvings—further validation of its robustness and adaptability is necessary. Future work will focus on expanding the range of heritage scenarios and component types to build a more diversified and representative damage identification dataset, enhancing the method’s generalizability and effectiveness in fine-grained damage detection and structural conservation. This research is primarily based on RGB-D imagery and point clouds. Future developments may incorporate multimodal data such as thermal infrared or hyperspectral imagery, which can reveal hidden cracks or differentiate between restoration materials. Integrating these modalities would enable more comprehensive and accurate damage diagnostics.

Additionally, by merging RGB-D data captured from multiple viewpoints, a more complete point cloud model could be constructed, thereby improving the accuracy of 3D coordinate and dimension measurements. Advancements in depth image inpainting techniques will also be explored to address issues of missing depth data caused by occlusion or reflective surfaces. Furthermore, building an error compensation model that dynamically adjusts based on the spatial location and depth of the damage within the image may enhance measurement precision. In future research, damage localization and analysis could be extended to multi-component coordination, where known damage information from a single component is integrated into finite element analysis for a comprehensive structural health assessment. Moreover, the proposed techniques may be applied in long-term HBIM-based monitoring systems for historic buildings. By periodically scanning and analyzing key components, a damage evolution database can be established, providing a scientific basis for preventive conservation strategies and valuable data to assess the response of historical architecture to environmental stressors such as climate change.