Application Research on General Technology for Safety Appraisal of Existing Buildings Based on Unmanned Aerial Vehicles and Stair-Climbing Robots

Abstract

Structure detection (SD) has emerged as a critical technology for ensuring the safety and longevity of infrastructure, particularly in housing and civil engineering. Traditional SD methods often rely on manual inspections, which are time-consuming, labor-intensive, and prone to human error, especially in complex environments such as dense urban settings or aging buildings with deteriorated materials. Recent advances in autonomous systems—such as Unmanned Aerial Vehicles (UAVs) and climbing robots—have shown promise in addressing these limitations by enabling efficient, real-time data collection. However, challenges persist in accurately detecting and analyzing structural defects (e.g., masonry cracks, concrete spalling) amidst cluttered backgrounds, hardware constraints, and the need for multi-scale feature integration. The integration of machine learning (ML) and deep learning (DL) has revolutionized SD by enabling automated feature extraction and robust defect recognition. For instance, RepConv architectures have been widely adopted for multi-scale object detection, while attention mechanisms like TAM (Technology Acceptance Model) have improved spatial feature fusion in complex scenes. Nevertheless, existing works often focus on singular sensing modalities (e.g., UAVs alone) or neglect the fusion of complementary data streams (e.g., ground-based robot imagery) to enhance detection accuracy. Furthermore, computational redundancy in multi-scale processing and inconsistent bounding box regression in detection frameworks remain underexplored. This study addresses these gaps by proposing a generalized safety inspection system that synergizes UAV and stair-climbing robot data. We introduce a novel multi-scale targeted feature extraction path (Rep-FasterNet TAM block) to unify automated RepConv-based feature refinement with dynamic-scale fusion, reducing computational overhead while preserving critical structural details. For detection, we combine traditional methods with remote sensor fusion to mitigate feature loss during image upsampling/downsampling, supported by a structural model GIOU [Mathematical Definition: GIOU = IOU − (C − U)/C] that enhances bounding box regression through shape/scale-aware constraints and real-time analysis. By siting our work within the context of recent reviews on ML/DL for SD, we demonstrate how our hybrid approach bridges the gap between autonomous inspection hardware and AI-driven defect analysis, offering a scalable solution for large-scale housing safety assessments. In response to challenges in detecting objects accurately during housing safety assessments—including large/dense objects, complex backgrounds, and hardware limitations—we propose a generalized inspection system leveraging data from UAVs and stair-climbing robots. To address multi-scale feature extraction inefficiencies, we design a Rep-FasterNet TAM block that integrates RepConv for automated feature refinement and a multi-scale attention module to enhance spatial feature consistency. For detection, we combine dynamic-scale remote feature fusion with traditional methods, supported by a structural GIOU model that improves bounding box regression through shape/scale constraints and real-time analysis. Experiments demonstrate that our system increases masonry/concrete assessment accuracy by 11.6% and 20.9%, respectively, while reducing manual drawing restoration workload by 16.54%. This validates the effectiveness of our hybrid approach in unifying autonomous inspection hardware with AI-driven analysis, offering a scalable solution for SD in housing infrastructure.

1. Improving Identification Technology Through Integrating Footage from an Unmanned Aerial Vehicle (Dajiang Technologies Inc., Shen Zhen, Guangdong, China) and Stair-Climbing Robot (CABRiolet Inc., Beijing, China)

The footage obtained from unmanned aerial vehicle (UAV) images to detect residential building clusters was collected in Kaihua, Zhejiang Province, and urban areas in Guizhou Province, China, and photos are shown in Figure 1. The dataset is in PASCAL VOC format and contains 492 images. The labeling tool was used to accurately label the Civil Architecture and Industrial Architecture classes of the residential buildings by expert teams from Zhejiang Provincial Construction Engineering Quality Inspection Station Co., Ltd., Hangzhou, P. R. China and Zhejiang Dahuo Inspection Co., Ltd, Hangzhou, P. R. China. During the labeling process, boxes with sizes of 70 × 70 and 50 × 50 were used to label the C and I classes, respectively, with 10,500 targets labeled for each class. These datasets were used in the experiments conducted in this project.

Figure 1. Examples of UAV photos and videos obtained in the survey.

1.1. DasViewer Model

Figure 2 shows an overall detection block diagram of the DasViewer building [1,2,3,4]. DasViewer software (ZhiYing R10) is a professional building modeling and analysis software independently invented in China, capable of automatic data processing, 3D modeling, image feature extraction, and analysis (Instruction manual reference: https://www.daspatial.com.cn/dasviewer, accessed on 1 January 2024.). Using the DasViewer model combined with advanced target detection technology, we propose an efficient and accurate method for detecting and identifying buildings. The method fully utilizes the broad field of view and the detailed information obtained from high-altitude photography. As shown in Figure 2, the attention mechanism calculates attention weights to enhance the ability to focus on individual buildings, effectively addressing the problems of target occlusion and visual confusion, thereby improving the accuracy of single target building detection in drone aerial photos. The aerial photos are enhanced to improve the clarity and contrast of the building boundaries, reduce the impact of complex environments and changes in lighting on detection, and increase accuracy. The adaptive pooling method is adopted to improve the spatial resolution of detection, capture smaller building details and boundaries, and enhance sensitivity and accuracy. Furthermore, the SMU activation function is used to suppress background noise, enhance building edge details, and improve clarity and stability. A high-definition visible light camera and an infrared thermal imager (Beijing Zhibolian Technology Co. Ltd, Bejing, China) are equipped, as shown in Figure 1. The high-definition camera has a resolution of up to 40 million pixels and can conduct a high-precision physical examination of the exterior walls of buildings from a distance of 10 m. The infrared thermal imager captures hidden dangers that are invisible to the naked eye by measuring the temperature of the wall related to sunlight exposure.

Figure 2. DasViewer diagrammatic figure.

1.2. Technology Acceptance Model (TAM) Attention Mechanism

The TAM attention mechanism was first proposed and used in natural language processing and machine translation for aligning text, and good results have been achieved. In the field of computer vision, some researchers have explored methods of using the attention mechanism in vision and convolutional neural networks to improve network performance. The basic principle of the attention mechanism is quite simple: the importance of different features, which can be different channels or different positions, differs in each layer of the network. The subsequent layers need to pay attention to the important information and suppress unimportant information (Figure 3, quoted from 2017 SENet [Squeeze-and-Excitation Network]). For example, in gender classification, greater emphasis should be placed on features such as hair length, which are more indicative of gender, rather than on features like waist width and height-to-head ratio, which are less relevant [5,6].

Figure 3. Algorithm flowchart of the technology acceptance model attention mechanism.

A flowchart of TAM is shown in Figure 4. The TAM consists of three parallel branches: the first two branches focus on cross-dimensional interactions between channels and spatial dimensions, while the third branch is similar to a convolutional attention module, concentrating on building spatial attention. The outputs of these three branches are aggregated through simple averaging. Among them, the triplet attention mechanism includes three branches, which calculate the attention weights between different dimensions to capture the dependencies across channels and spatial dimensions. The first two branches use rotation operations to strengthen the connection between channels and spatial dimensions. The model is suitable for use in processing aerial/capture images from a UAV or climbing robot, and it can precisely focus on the nodes and other targets of buildings, improving the detection performance in complex backgrounds and high-altitude perspectives.

Figure 4. Flowchart of the technology acceptance model.

1.3. Identification Parameters

1.3.1. Parameter: Preliminary Inspection of Current Structural Condition [7,8,9]

Using a UAV and a stair-climbing robot equipped with infrared imaging devices, the building structure was investigated. Relevant data and information about its expansion, renovation, and usage history were obtained and preliminarily analyzed.

1.3.2. Parameter: Structural Material Strength Testing [10,11]

Significant rebound force occurs when obtaining information about the strength of a structural material, and the maximum rebound value of a common rebound tester (Beijing Zhibolian Technology Co. Ltd, Bejing, China) reaches 60 MPa. According to the specification, the final value after calculation based on the carbonization depth reached 60 MPa, which is much heavier than the weight of the UAV and plaque robot. At the same time, the detection requirements for this parameter are relatively high. Detection needs to be performed horizontally or vertically, and in full contact with the detection surface. Therefore, after several attempts, we finally decided to use an intelligent device to complete the process.

1.3.3. Parameter: Component Cross-Sectional Size Inspection [12,13,14,15,16,17]

This parameter was detected and investigated through point cloud scanning using infrared imaging equipment carried by the UAV and stair-climbing robot. Under good lighting conditions, the collected data were accurate and effective, but precision was superior for individual components.

1.3.4. Parameter: Inspection of Reinforcement Configuration [18]

This parameter was detected and investigated using a UAV and a stair-climbing robot equipped with ultrasonic steel bar scanning devices. Based on the principle that the propagation speed of ultrasonic waves in concrete and steel bars is inconsistent, a steel bar layout map was formed, and the position of the main steel bars in the structure was accurately located.

1.3.5. Parameter: Appearance Quality Inspection [19,20,21]

A UAV and a stair-climbing robot were employed to comprehensively detect and investigate defects and cracks using ultrasonic concrete flaw detectors and crack detection devices. Based on the principle that the propagation speed of ultrasonic waves in concrete and air is different, the wave speed was combined with the wave speed of the entire concrete and entered into the formula for comparison and calculation. This enabled defect conditions to be accurately identified and measurements of the width and depth of cracks to be conducted.

1.3.6. Parameter: Calculation of Structural Seismic Bearing Capacity [20]

YJK software (structural modeling software, YCAD) was used to conduct a structural bearing capacity calculation of the entire building and obtain a corresponding structural model and calculation data. The calculations were conducted in accordance with the GB 50011 standard [21,22] of the Ministry of Housing and Urban–Rural Development of the People’s Republic of China, and the final value of the structural seismic bearing capacity was analyzed.

The equipment used to determine the above parameters is shown in Figure 5.

Figure 5. Images of the main equipment used in identification and testing.

2. Evaluation Indicators

Average precision (AP) is a metric commonly used to evaluate the performance of object detection algorithms. The value of AP is the area enclosed by the PR curve and the x-axis and y-axis. The mean average precision (M-AP) is used in multiple object detection categories, and it is obtained by summing the AP values corresponding to each category and calculating the average.

The model was evaluated on the test set, and the Precision and Recall corresponding to each predicted bounding box were calculated. In simple terms, each image in the test dataset was consecutively sent to the object detection [4,23], and the corresponding prediction results were obtained. Then, in the prediction results, each predicted object was traversed, and in accordance with the chronological order, the current precision (Precision) and recall rate (Recall) were calculated for each predicted bounding box obtained. The formulas used to calculate Precision and Recall are as follows,

Precision = TP/(TP + FF)

(1)

Recall = TP/(TP + FN) = TP/(All Ground Truths)

(2)

where TP represents the number of correctly predicted targets, FP represents the number of incorrectly predicted targets, FN represents the number of targets that were not detected, and GP represents the total number of actual targets.

3. Comparative Analysis of Results

3.1. Comparison and Analysis of Experimental Results

3.1.1. Comparison Between Image Collection Modeling and Modeling After Manual Image Collection

Through an in-depth analysis and processing of the flight/capture images from the UAV and stair-climbing robot, the performance indicators of the UAV, stair-climbing robot, and manual image collection for the Building_W category and the Building_P category were compared, and are presented in the line chart in Figure 6. These indicators included AP, LAMR, Precision, Recall, and F1 value, which comprehensively reflected the performance of the three models in the target detection task. The tests included multiple acquisition methods such as MODEL-A, MODEL-R, and MODEL-M to create models [24]. These models are all based on deep learning target detection algorithms and have their own characteristics and advantages.

Figure 6. Line chart showing the comparative experiment results of the three models.

The detection performances of the UAV and the stair-climbing robot in the aerial images of the Building_W category and the Building_P category are shown in Table 1 [25]. For the Building_W category, the MODEL series models generally exhibited high Precision, ranging from 79.52% to 83.89%, demonstrating the reliable accuracy of the models in identifying buildings. Recall was relatively low, indicating that the models may have omissions when detecting all buildings. The F1 value, as the harmonic mean of Precision and Recall, ranged from 0.47 to 0.69, indicating that the established model provided a relatively balanced overall performance. The Building_P model provided a high performance, and its AP and F1 values were at the leading level. For the Building_P category on the other side, the detection difficulty significantly increased. Due to lighting issues, the series models still have some detection capabilities. The MODEL-A and MODEL-R models performed particularly well in model establishment and collection, reaching a basic level compared to MODEL-M, and their AP and F1 values were higher.

Table 1. Comparative experiment results of the three models.

Model Series	Category	AP	LAMR	Precision	Recall	FI
MODEL-A (UAV)	Building_W	61.12%	0.95	82.88%	28.72%	0.47
	Building_P	35.88%	0.96	83.89%	88.50%	0.12
MODEL-R (Ladder-climbing robot)	Building_W	69.24%	0.95	81.87%	38.70%	0.56
	Building_P	54.81%	0.98	82.46%	10.52%	0.19
MODEL-M (Manually established)	Building_W	52.23%	0.89	79.52%	36.24%	0.69
	Building_P	32.42%	0.87	80.14%	17.52%	0.54

Note. In the provided comparison results of the MODEL models, the bold data represent the best performance of a certain metric (AP, LAMR, Precision, Recall, F1) in a specific category (Building_W or Building_P).

3.1.2. Comparison Between Strength Data Collection and Manual Collection of Strength Data

The strength data collection module is shown in Figure 7, which presents the results of different detection parameters during the data collection process when the stair-climbing robot was equipped with the device. These parameters included mortar rebound (brick strength detection), brick rebound (mortar strength detection), concrete rebound (compressive strength detection of concrete components), and the comparison with the manual test. These parameters were set to enhance the ability of the stair-climbing robot to collect data of different scales and types during its operation, thereby optimizing the target detection and collection performance. The mechanism of the stair-climbing robot performed particularly well with respect to the overall effect, and it was able to fully integrate different types of information on different scales and to help the device accurately capture key targets during data collection [26]. At the same time, through adaptive adjustment of weight allocation, the stair-climbing robot was able to focus more on important features (such as high-position components), further improving the accuracy and stability of target detection.

Figure 7. Line chart showing comparison between strength data Collection.

All tests were conducted strictly in accordance with the current national standards. The climbing robot was equipped with various settings, and it was used in the same horizontal direction to perform a mechanical rebound force on vertical components. Data were collected during the operation, and the operational performance was significantly improved [27]. In particular, the equipment carried by the climbing robot provided an excellent performance when the climbing robot was above the ergonomic position. The deviation of the same type of detection values was between 1.619 and 5.095, as shown in Table 2. The detection standard deviation of the equipment carried by the climbing robot tower was greater than that of manual detection. However, the collected detection data performance was also relatively low, indicating that more experiments are required to improve the stability and effectiveness of the intelligent detection by the equipment carried by the climbing robot, or the final estimated value.

Table 2. Comparison of intensity data collection experiment performances.

Detecting Parameter	Category	Detection Zone										Standard Deviation
		1	2	3	4	5	6	7	8	9	10
Mortar rebound	Using a stair-climbing robot to carry equipment for detection (MPa)	12.4	16.5	14.2	12.2	10.5	10.2	10.5	12.5	12.5	11.5	1.911
	Manual Testing (MPa)	22.2	24.5	22.5	22.2	21.0	22.5	19.3	20.2	22.5	22.5	1.446
Brick Rebound	Using a stair-climbing robot to carry equipment for detection (MPa)	24.5	25.8	26.5	27.5	29.6	30.2	30.4	32.1	32.2	32.5	2.881
	Manual Testing (MPa)	40.2	38.2	39.2	38.2	35.4	39.2	40.2	40.9	39.6	39.4	1.539
Concrete Rebound	Using a stair-climbing robot to carry equipment for detection (MPa)	28.3	31.5	15.6	16.7	18.5	19.5	20.2	21.3	18.8	22.6	5.021
	Manual Testing (MPa)	35.5	34.8	32.2	30.5	28.0	29.0	30.2	29.5	29.5	30.5	2.467

Note. In the provided detection parameter comparison results, the tilted data represent abnormal data of a certain indicator at a specific point within a certain category. This test only compares the measured values, so no units are provided.

3.1.3. Collection and Comparison of Other Parameters with Manually Collected Data of Component Dimensions and Crack Widths

A comparison of the detection parameters is shown in Figure 8. In this paper, the proposed method of using UAVs or climbing robots in conjunction with other mainstream manual detection methods was compared with the experimental results after collecting data by UAVs or climbing robots. The point-line graph clearly demonstrates the differences in performance among the various detection types. Since UAV photography images usually contain complex background information, multi-scale targets, and special visual features at high-altitude perspectives, it is particularly important to select appropriate detection components. As can be seen from the figure, compared with target detection methods based on parameters using a UAV and climbing robot, UAVs and climbing robots demonstrated higher performance in building detection tasks. This was mainly due to the advantages of the UAV and climbing robot in collecting real-time and accurate infrared light [28], as well as the optimizations made by UAVs in the characteristics of aerial photography images. At the same time, the performance of the UAV and climbing robot was better when identifying and detecting the quality of building exteriors. This indicates that the self-mounted fusion of an infrared light instrument with a UAV or climbing robot further enhances the ability of these technologies in detection and identification image processing.

Figure 8. Line chart comparing length parameter data collection.

As shown in Table 3, there were no significant differences between the results for the different detection methods in measuring the size and crack widths. Specifically, the crack detection values detected by the equipment carried by the UAV were 18.50% higher than those detected by manual methods, while the detection values of the equipment carried by the robot were 25.58% higher than those detected using manual methods. In addition, the equipment carried by the UAV and robot performed well with key indicators such as crack depth and size detection, ensuring the accuracy and completeness of target detection [29]. In terms of annotation categories and flexible operation, both detection methods also demonstrated strong performance. This feature enabled the UAV and climbing stairs robot to perform well in detection tasks, providing an effective solution for related applications.

Table 3. Comparison of other parameter data collected.

Detecting Parameter	Category	Detection Zone 1	Detection Zone 2	Detection Zone 3	Detection Zone 4	Detection Zone 5	Detection Zone 6	Detection Zone 7	Detection Zone 8	Detection Zone 9	Detection Zone 10
Length of concrete column (design: 500 × 500) (mm)	Using a stair-climbing robot to carry equipment for detection	2	3	2	3	3	2	3	0	3	2
	Using unmanned aircraft to carry equipment for detection	3	2	2	3	3	3	2	2	2	2
	Manual testing	3	3	3	3	3	2	1	2	3	3
(After comparing the measured values of the same survey area) Standard deviation		0.577	0.577	0.577	0.000	0.000	0.577	1.000	1.155	0.577	0.577
Crack width of masonry components (mm)	Using a stair-climbing robot to carry equipment for detection	0.825	0.728	1.585	0.885	0.375	2.584	0.825	0.854	0.925	1.854
	Using unmanned aircraft to carry equipment for detection	0.724	0.774	1.425	0.754	0.348	2.554	0.458	1.258	0.778	1.725
	Manual testing	0.558	0.578	1.224	0.625	0.421	2.087	0.527	0.847	0.688	1.555
(After comparing the measured values of the same survey area) Standard deviation		0.135	0.102	0.181	0.130	0.037	0.279	0.195	0.235	0.120	0.150

Note. In the comparison experiment results of the provided detection parameters, in this test, only the measured values were compared. The calculated estimated values were not included. All units are in millimeters.

3.2. Technology Application

The experimental results for the detection task completed by the two machines are shown in Figure 6, Figure 7, Figure 8 and Figure 9, where detailed comparisons of the models and their performances with intensity parameters, images, size parameters, crack width, and their practical effectiveness during the identification and detection process are made. The characteristics of the UAV and climbing robot differ when processing images or data, and there is room for improving the intensity parameters, details, and accuracy. Both the UAV and climbing robot performed particularly well in identifying parameters in old buildings, especially in cases where no original design drawings were used for modeling [27]. The built model not only accurately detected the center position of buildings and components, but it also showed strong applicability in dealing with diverse shapes and visually confusing objects. However, in some complex scenarios, such as when the building density was high or the lighting or wind speed conditions were poor, applicability still presented certain challenges.

Figure 9. Line graph showing comparison of crack parameter data collected.

4. Conclusions and Outlook

We applied a UAV and stair-climbing robot to improve identification technology and achieved significant improvements in building identification detection tasks during image collection and model establishment. Through their application, the problems of target occlusion and visual confusion were effectively handled. Through comparative tests, deviations of all parameters were found to be within the ideal range. Improvements were observed in adapting to morphological changes and varying scene backgrounds during size and crack depth detection. Use of a UAV and stair-climbing robot enabled detailed features to be detected, further improving the detection and identification effect. At the same time, the application of infrared scanning technology improved the expression ability, generalization ability, and applicability of the neural network. The experimental results showed that the overall detection efficiency increased by more than 30%. The efficiency of drawing restoration was also greatly improved, with the workload reduced from 3 people and 7 days to 1 person and 7 days. These results verify the effectiveness and superiority of the new identification method in detection and identification application tasks [24].

In future research, we will address several critical challenges identified during our field deployments. Specifically. we will discuss potential improvements, such as optimizing the force control algorithm of the UAV and Stair-Climbing Robot, adding a shock absorber, or using a strategy of taking multiple measurements and averaging them to reduce variability.

To address these limitations, we will explore hybrid approaches combining advanced attention mechanisms with reinforcement learning methods specifically optimized for resource-constrained operation. This includes developing adaptive detection models that dynamically balance precision and recall based on remaining battery life and mission priorities. Furthermore, we will investigate tight integration between real-time monitoring drones and climbing robots to create synergistic systems where each platform compensates for the other’s operational weaknesses, enabling more efficient large-scale structural assessment while maintaining detection accuracy across diverse urban environments.

By explicitly addressing these performance–operation tradeoffs, we aim to develop more robust and practical solutions for urban renewal applications, ultimately advancing toward fully autonomous building inspection systems capable of precise, high-quality structural assessment under real-world constraints.