Automated Damage Detection on Concrete Structures Using Computer Vision and Drone Imagery

: The manual inspection of concrete structures, such as tall buildings, bridges, and huge infrastructures can be time-consuming and costly, and damage assessment is a crucial task that requires the close-range inspection of all surfaces. The proposed system uses computer vision model to identify various types of damages on these structures. The computer vision model and was trained on a large dataset of drone footage, which was annotated manually to ensure accuracy. The model was then tested on new data, and the results showed that it could accurately detect and identify structural damage on concrete structures with a 94% accuracy. The system is much faster and more efficient than manual inspection, reducing the time and cost required for damage assessment. The proposed system has the potential to revolutionize the way we perform damage assessment on concrete structures. It can help to preserve and protect these valuable assets by enabling the early detection of damage and facilitating timely repairs.


Introduction
The manual inspection of concrete structures, such as tall buildings, bridges, and huge infrastructures, is a time-consuming and risky process for human employees.Drones with sensor camera nodes have shown potential in gathering close-range footage, but the problem lies in rapidly analyzing enormous volumes of data to detect and diagnose structural deterioration.This study deals with these challenges by presenting an Internet of Things (IoT), computer-vision-and deep-learning-based automated solution.The primary issue addressed in this research is the requirement for a more efficient and reliable way of identifying structural damage on concrete structures.
The traditional manual inspection technique is time-consuming and expensive, making timely repairs and maintenance impossible.As a result, an automated solution is necessary to speed up the damage assessment process while reducing dangers to human personnel.The proposed system focuses on detecting various types of damage, such as cracks, Alkali-Silica Reactions (ASRs), concrete degradation, and others, on concrete structures using drone-captured video footage [1,2].The system's scope includes developing a Convolutional Neural Network (CNN) architecture tailored to this specific task and implementing a seamless process for automatically obtaining video data from drones.
The primary objective of this work is to create and implement an automated damage detection system capable of identifying structural damage on concrete structures in an efficient and accurate manner.The technology intends to expedite the inspection process by utilizing IoT, computer vision, and deep learning techniques, enabling proactive maintenance and preservation activities.The novelty of the proposed system is a custom-designed CNN architecture that is optimized for detecting damage on concrete structures and a system architecture based on IoT to automatically capture data and perform analysis and reporting.The performance of the proposed automated damage detection system was evaluated using a diverse dataset of drone-captured video footage containing various types of damage on concrete structures.The CNN architecture demonstrated impressive results, achieving an accuracy of 94% in correctly identifying different types of structural damage.
The approach involves capturing close-range footage of the infrastructure with drones and processing the footage using a computer vision model to identify and classify damage.The proposed method can provide an efficient and cost-effective way to detect damage to concrete structures and help preserve these important historical structures for future generations.The working of the system is shown in Figure 1.
by utilizing IoT, computer vision, and deep learning techniques, enabling proact maintenance and preservation activities.The novelty of the proposed system is a custo designed CNN architecture that is optimized for detecting damage on concrete structu and a system architecture based on IoT to automatically capture data and perform ana sis and reporting.The performance of the proposed automated damage detection syst was evaluated using a diverse dataset of drone-captured video footage containing vario types of damage on concrete structures.The CNN architecture demonstrated impress results, achieving an accuracy of 94% in correctly identifying different types of structu damage.
The approach involves capturing close-range footage of the infrastructure w drones and processing the footage using a computer vision model to identify and class damage.The proposed method can provide an efficient and cost-effective way to det damage to concrete structures and help preserve these important historical structures future generations.The working of the system is shown in Figure 1.

Related Work
This comprehensive analysis [3] explores into the critical issue of damage detecti in civil engineering, emphasizing the transformational potential of Computer Vision a Deep Learning algorithms.The research [4] presents a Deep Convolutional Neural N work-based Damage Locating (DCNN-DL) approach for steel frame inspection that o performs existing techniques with a 99.3% accuracy.The approach uses the DenseNet chitecture to properly identify and detect damaged regions, providing a fast real-time lution for visual damage evaluation in civil structures.The research in [5] investigates use of computer vision algorithms in conjunction with remote cameras and unmann aerial vehicles (UAVs) for non-contact civil infrastructure evaluation.The study in [6] p sents a unique structural identification framework for bridge health monitoring th makes use of computer-vision-based measures.It employs a novel damage indicator displacement unit influence surface, and successfully identifies and localizes simulat damage on a large-scale bridge model, demonstrating its use in structural health evalu tion.The thorough study in [7] addresses gaps in the current literature on computersion-based crack diagnosis for civil infrastructures by providing a complete evaluation qualitative and quantitative methodologies, including deep-learning-based approache

Related Work
This comprehensive analysis [3] explores into the critical issue of damage detection in civil engineering, emphasizing the transformational potential of Computer Vision and Deep Learning algorithms.The research [4] presents a Deep Convolutional Neural Networkbased Damage Locating (DCNN-DL) approach for steel frame inspection that outperforms existing techniques with a 99.3% accuracy.The approach uses the DenseNet architecture to properly identify and detect damaged regions, providing a fast real-time solution for visual damage evaluation in civil structures.The research in [5] investigates the use of computer vision algorithms in conjunction with remote cameras and unmanned aerial vehicles (UAVs) for non-contact civil infrastructure evaluation.The study in [6] presents a unique structural identification framework for bridge health monitoring that makes use of computer-vision-based measures.It employs a novel damage indicator, a displacement unit influence surface, and successfully identifies and localizes simulated damage on a large-scale bridge model, demonstrating its use in structural health evaluation.The thorough study in [7] addresses gaps in the current literature on computer-vision-based crack diagnosis for civil infrastructures by providing a complete evaluation of qualitative and quantitative methodologies, including deep-learning-based approaches.
Despite extensive research into image-based damage identification and quantification, this technology is still in its early phases, with limitations and gaps for further investigation.Despite efforts in improving the reliability of image-based approaches, attaining total automation in damage assessment and categorization remains a substantial task.

System Design
The proposed system involves the use of a drone to capture image data, which are then fed to an object detection model to identify different types of structural damage in concrete structures.In this section, we provide a detailed methodology for building and deploying the system, which involves collecting images of cracks in buildings, labeling and training the model used, and deploying the model using a Python 3.x script.

Dataset Collection
The first step in building the system is to collect images of cracks in buildings.These images are taken using a drone, which captures high-resolution images of the building's surface.The images are then annotated with labels indicating the type of damage using the Roboflow platform.The labeling process involves drawing bounding boxes around the damaged area and assigning them a label indicating the type of damage, such as cracks, ASR, or concrete degradation.
Different images of cracks were collected.The images are sized to 800 × 800 pixels which is a standard transformation applied before training.The image dataset is then split up into training, testing, and validating groups: With the use of the Roboflow platform's rectangular "bounding box" tool, each picture in the dataset is labelled for object detection.The labelling for the various photos is shown in the Figure 2:

System Design
The proposed system involves the use of a drone to capture image data, w then fed to an object detection model to identify different types of structural da concrete structures .In this section, we provide a detailed methodology for build deploying the system, which involves collecting images of cracks in buildings, and training the model used, and deploying the model using a Python 3.x script.

Dataset Collection
The first step in building the system is to collect images of cracks in building images are taken using a drone, which captures high-resolution images of the bu surface.The images are then annotated with labels indicating the type of damag the Roboflow platform.The labeling process involves drawing bounding boxes the damaged area and assigning them a label indicating the type of damage, cracks, ASR, or concrete degradation.
Different images of cracks were collected.The images are sized to 800 × 80 which is a standard transformation applied before training.The image dataset is th up into training, testing, and validating groups:

Model Training
The annotated data are then used to train the object detection model.The m trained using a deep learning algorithm, YOLO.
The trained model is deployed using a Python script, which receives a live stream and runs the object detection model to detect the cracks.The Python scr the Model API to query the hosted version of the model and return the results.provides a simple interface for sending images or video to the model and recei output in a standardized format.The methodology of system is presented in the F

Model Training
The annotated data are then used to train the object detection model.The model is trained using a deep learning algorithm, YOLO.
The trained model is deployed using a Python script, which receives a live camera stream and runs the object detection model to detect the cracks.The Python script uses the Model API to query the hosted version of the model and return the results.The API provides a simple interface for sending images or video to the model and receiving the output in a standardized format.The methodology of system is presented in the Figure 3.
The selection of hardware components plays a crucial role in this research as they enable the deployment and testing of the computer vision model.Specifically, a drone and a camera module are utilized to facilitate the implementation of the model.The Figure 4 below illustrates the setup of the drone used in this research.The setup of the drone, as depicted in the figure, showcases the integration of the camera module and other necessary components to ensure seamless data capture and transmission.The system incorporates several essential hardware components for its operation.It utilizes a Quadcopter Drone Kit, WiFi Camera Module, a Roboflow account, and a Python development environment.

Results and Discussion
In this section, the results of experiments using different object detection models for detecting physical damage in concrete structures has been discussed.The three different models, YOLOv8, YOLOv7, and YOLOv5, are used to evaluate the performance of the system on a test dataset of annotated images for the field of concrete structures monitoring and protection.
The performance of the YOLOv8, YOLOv7, and YOLOv5 models has been evaluated on the test dataset of annotated images.The models were trained using the dataset from the Roboflow platform.As transfer learning is used in this research, the following tables, Table 1 and Table 2, show the pre-trained YOLO models used and the training settings.The system incorporates several essential hardware components for its oper utilizes a Quadcopter Drone Kit, WiFi Camera Module, a Roboflow account, and a development environment.

Results and Discussion
In this section, the results of experiments using different object detection mo detecting physical damage in concrete structures has been discussed.The three d models, YOLOv8, YOLOv7, and YOLOv5, are used to evaluate the performanc system on a test dataset of annotated images for the field of concrete structures mo and protection.
The performance of the YOLOv8, YOLOv7, and YOLOv5 models has been ev on the test dataset of annotated images.The models were trained using the datas the Roboflow platform.As transfer learning is used in this research, the following Table 1 and Table 2, show the pre-trained YOLO models used and the training sett The system incorporates several essential hardware components for its operation.It utilizes a Quadcopter Drone Kit, WiFi Camera Module, a Roboflow account, and a Python development environment.

Results and Discussion
In this section, the results of experiments using different object detection models for detecting physical damage in concrete structures has been discussed.The three different models, YOLOv8, YOLOv7, and YOLOv5, are used to evaluate the performance of the system on a test dataset of annotated images for the field of concrete structures monitoring and protection.
The performance of the YOLOv8, YOLOv7, and YOLOv5 models has been evaluated on the test dataset of annotated images.The models were trained using the dataset from the Roboflow platform.As transfer learning is used in this research, the following tables, Tables 1 and 2, show the pre-trained YOLO models used and the training settings.The evaluation metrics used were precision, recall, and F1 score.Equations ( 1)-( 4) [8] were used to evaluate the performance of machine learning models and measure their accuracy, precision, recall, F1 score, and latency: Accuracy = (TP + TN)/(TP + TN + FP + FN) (1) Recall = TP/(TP + TN) Table 3 shows the results of the evaluation for each model.As shown in Table 1, the YOLOv8 model achieved the highest performance in terms of precision, recall, and F1 score.The model achieved a precision of 0.93, recall of 0.85, and F1 score of 0.89.The YOLOv7 model also performed well with a precision of 0.92, recall of 0.81, and F1 score of 0.86.The YOLOv5 model achieved a precision of 0.90, recall of 0.75, and F1 score of 0.82.
The following Table 4 compares the mAP scores for each class across the YOLOv8, YOLOv7, and YOLOv5 models: As shown in the table, the YOLOv8 model has the highest mAP score for all classes except for "Heaving cracks", where the YOLOv7 model performs slightly better.The YOLOv7 model also shows good performance for "Crazing and Crusting Cracks", "Settling cracks", and "Corrosion of Reinforcement".The YOLOv5 model generally shows lower mAP scores for all classes compared to the other two models, but still performs relatively well for "Crazing and Crusting Cracks" and "Corrosion of Reinforcement".The following Table 5 compares the inferencing time for all three models.As shown in the table, the YOLOv8 model has the highest mAP score and a relatively fast inference time, making it the best overall option for detecting structural damages.The YOLOv7 model has a moderate mAP score but a slightly slower inference time compared to the YOLOv8 model.The YOLOv5 model has the fastest inference time but the lowest mAP score, making it a suitable option for applications where speed is a priority over accuracy.The following Figure 5 depict the result from YOLOv8 model.
YOLOv7 model also shows good performance for "Crazing and Crusting Cracks", "Settling cracks", and "Corrosion of Reinforcement".The YOLOv5 model generally shows lower mAP scores for all classes compared to the other two models, but still performs relatively well for "Crazing and Crusting Cracks" and "Corrosion of Reinforcement".The following Table 5 compares the inferencing time for all three models.As shown in the table, the YOLOv8 model has the highest mAP score and a relatively fast inference time, making it the best overall option for detecting structural damages.The YOLOv7 model has a moderate mAP score but a slightly slower inference time compared to the YOLOv8 model.The YOLOv5 model has the fastest inference time but the lowest mAP score, making it a suitable option for applications where speed is a priority over accuracy.The following Figure 5 depict the result from YOLOv8 model.

Conclusions
This research demonstrates the effectiveness of using drone-captured imagery and computer vision techniques for the inspection of physical damage in in concrete structures such as buildings, bridges etc.By employing object detection models such as YOLOv8, YOLOv7, and YOLOv5, various types of damage, including plastic shrinkage cracks, crazing and crusting cracks, settling cracks, expansion cracks, heaving cracks, overloading cracks, and corrosion of reinforcement, have been successfully identified.The evaluation of the models has shown promising results, with high mAP scores across different classes.
This research has highlighted the potential of leveraging computer vision and drone technology in damage assessment, providing a safer, cost-effective, and efficient alternative to traditional manual inspections.By automating the detection process, we reduce the need for manual evaluation, which can be time-consuming and prone to human error.The integration of these technologies allows for comprehensive and detailed inspections, facilitating early detection and timely intervention to mitigate further damage to concrete structures.
Author Contributions: T.M., S.T. and R.K.D., conceptualized the idea for this manuscript.T.M. provided the resources.S.T. designed and developed the hardware for data acquisition.R.K.D. carried

Conclusions
This research demonstrates the effectiveness of using drone-captured imagery and computer vision techniques for the inspection of physical damage in in concrete structures such as buildings, bridges etc.By employing object detection models such as YOLOv8, YOLOv7, and YOLOv5, various types of damage, including plastic shrinkage cracks, crazing and crusting cracks, settling cracks, expansion cracks, heaving cracks, overloading cracks, and corrosion of reinforcement, have been successfully identified.The evaluation of the models has shown promising results, with high mAP scores across different classes.
This research has highlighted the potential of leveraging computer vision and drone technology in damage assessment, providing a safer, cost-effective, and efficient alternative to traditional manual inspections.By automating the detection process, we reduce the need for manual evaluation, which can be time-consuming and prone to human error.The integration of these technologies allows for comprehensive and detailed inspections, facilitating early detection and timely intervention to mitigate further damage to concrete structures.

•
2527 images used for training.• 2149 images used for validation.• 279 images used for testing.

•
2527 images used for training.• 2149 images used for validation.• 279 images used for testing.With the use of the Roboflow platform's rectangular "bounding box" tool, e ture in the dataset is labelled for object detection.The labelling for the various p shown in the Figure2:

Figure 2 .
Figure 2. Annotated images in the dataset.Bounding box for cracks detection.

Figure 2 .
Figure 2. Annotated images in the dataset.Bounding box for cracks detection.

Figure 3 .
Figure 3. Methodology of the system.

Figure 3 .
Figure 3. Methodology of the system.

Figure 3 .
Figure 3. Methodology of the system.

Figure 5 .
Figure 5. Inferencing result showing detection of cracks on wall.

Figure 5 .
Figure 5. Inferencing result showing detection of cracks on wall.

Table 3 .
Performance matrix of different YOLO models.

Table 4 .
Class-wise accuracy of different YOLO models.

Table 5 .
Accuracy and speed comparison of YOLO models.

Table 5 .
Accuracy and speed comparison of YOLO models.