Machine Learning-Based Concrete Crack Depth Prediction Using Thermal Images Taken under Daylight Conditions

Abstract

Concrete cracks can threaten the usability of structures and degrade the aesthetics of buildings. Furthermore, minor cracks can develop into large-scale cracks that may lead to structural failure when exposed to excessive external loads. In addition, the concrete crack width and depth should be precisely measured to investigate the effects of concrete cracks on the stability of structures. Thus, a nondestructive and noncontact testing method was introduced for detecting concrete crack depth using thermal images and machine learning. The thermal images of the cracked specimens were obtained using a constant test setup for several months under daylight conditions, which provided sufficient heat for measuring the temperature distributions of the specimens, with recording parameters such as air temperature, humidity, and illuminance. From the thermal images, the crack and surface temperatures were obtained depending on the crack widths and depths using the parameters. Four machine-learning algorithms (decision tree, extremely randomized tree, gradient boosting, and AdaBoost) were selected, and the results of crack depth prediction were compared to identify the best algorithm. In addition, data bias analysis using principal component analysis, singular value decomposition, and independent component analysis were conducted to evaluate the efficiency of machine learning.

1. Introduction

Concrete cracks at the exposed surfaces of concrete structures are caused by shrinkage, construction deficiencies, and deterioration of the structure with time. These cracks can threaten the stability of concrete structures and degrade the aesthetics of buildings. In addition, minor cracks can develop into large-scale cracks, which lead to structural failure when exposed to excessive external loads. The concrete crack width and depth should be precisely measured to investigate the effects of concrete cracks on the stability of structures. Numerous studies on detecting concrete width have been conducted, whereas studies on detecting concrete depth have rarely been conducted owing to many unknown factors [1,2]. Aggelis et al. proposed surface crack depth and repair evaluation using Rayleigh waves [1]. Slots 4 mm wide, whose depths were specifically 2–23 mm, were artificially made in laboratory-size concrete specimens to investigate the accuracy of the proposed technique. Lin and Wang detected concrete crack depths using ultrasonic shear horizontal waves [2].

The most common method for detecting concrete cracks is placing the detecting device in direct contact with the concrete surface. However, it is difficult to access every concrete surface area located in tall buildings and inaccessible places. Thus, new methods using thermography cameras have attracted the attention of field engineers. Sham et al. [3] verified that the concrete crack temperatures were higher than the intact concrete temperatures at the surface and investigated the radiative mechanism at the concrete crack through tests. The flash method in this study obtained thermal images after the concrete surface was heated for 3 ms, and can detect cracks from 0.5–1.0 mm wide. Omar et al. [4,5] proposed an automated method for detecting delamination in a concrete bridge deck using infrared thermography. In addition, an unmanned aerial vehicle (UAV) was used to take images from all views. Kodikara et al. [6] measured thermal diffusivity of soil with thermal images with a single fixed temperature end method in laboratory conditions. Andrade and Eduardo [7] proposed a methodology for detecting internal defects of fired ceramic tiles using thermal images and a multilayer perception network, the most widely used neural classifier. The three-layer was constituted of 72, 40, and 5 neurons in the input, hidden, and output layers, respectively. Bauer et al. [8] qualitatively and quantitively diagnosed façade defects using two kinds of thermography cameras with heat sources in the laboratory. Seo [9] detected initial crack and propagation using temperature changes in the thermal images and validated the detection method via strain gauge data. Chun and Hayashi [10] proposed a remote detecting method for concrete floating and delamination using thermal images. The thermal images were analyzed by machine learning with seven factors to detect the defects. Feroz and Abu Dabous [11] presented UAV-based remote sensing applications for bridge condition assessment, such as visual imagery, light detection, and ranging technology, and an infrared thermography camera. Mandirola et al. [12] proposed a UAV-based methodology for the inspection and damage assessment of bridges in the aftermath of a disaster, or in normal conditions.

There are limitations in quantitatively evaluating concrete cracks using thermal images, because of many unknown parameters such as crack shapes and concrete thermal characteristics. To solve this problem, machine learning was selected to investigate the relationships between unknown parameters and concrete defects. However, studies on detecting the existence of concrete defects and classifying the types of defects using machine learning have mainly been conducted based on photographs [13,14,15,16,17,18,19,20,21]. Recently, several studies have been conducted using thermal images and machine learning for predicting concrete crack depths [22,23,24,25]. Jeong et al. first introduced a crack depth prediction system using thermal images obtained from UAVs and machine learning. However, the accuracy of crack depth detection is relatively low because of the lack of thermal image data for machine learning [22]. Lee et al. applied the system to existing buildings to predict concrete crack depths, and machine learning was improved to reduce errors of prediction [23]. Bae et al. described a similar detection system for macrocrack depth (10–60 mm) using infrared thermography with three machine-learning algorithms (linear regression, random forest, and gradient boosting) implemented in Python 3.9 with 3639 thermal images [24,25]. However, the accuracy of crack depth detection needs to be greatly improved. Kim et al. used four machine-learning algorithms (multilayer perceptron, random forest, gradient boosting, and AdaBoost) to improve the accuracy of the system with 6940 thermal images [26]. The accuracies of all the algorithms, except the multilayer perceptron, were greatly improved; the maximum error was 3.1%. Moreover, the AdaBoost algorithm showed the best accuracy (98.96%) among the four machine-learning algorithms.

In this study, the accuracies of the other three machine-learning algorithms (decision tree, extremely randomized tree, and extreme gradient boosting) were compared to the best machine-learning algorithm (AdaBoost) in the previous study [26]. The amount of data for determining the most appropriate algorithm for assessing the concrete crack depths is equal to that in the previous study [26]. However, to increase the validation of machine learning, data bias, which was not conducted in the previous study, was also investigated to prevent overfitting. Finally, the performance indicators of the four machine-learning algorithms, considering data bias analysis, were described using three techniques that increased the efficiency and reduced the training duration methods. The differences between the algorithms and techniques were compared to suggest the best algorithm for predicting concrete crack depths.

2. Thermal Image Collection

2.1. Test Program

Concrete specimens with known cracks were artificially manufactured to collect the thermal image data for machine learning. The procedure for collecting thermal images and conducting machine learning to investigate the correlation between the crack depths and parameters is shown in Figure 1. This test was conducted at 1–4 pm, which is the typical working time for structural inspections; during this time, there is sufficient daylight for the specimen to show a significant temperature distribution at the crack surface. Numerous thermal images of artificially defective concrete specimens were obtained along with environmental parameters (illuminance, humidity, etc.). Machine learning was performed to train to analyze the data for predicting the crack depth.

2.2. Test Specimen

The test specimens were named according to the crack width (W) and depth (H), as listed in

Table 1. Name of test specimens.

Crack Depth (H) (mm)	Crack Width (W) (mm)
	5	8	10	12	15	20
10	-	-	C10-10	-	C15-10	C20-10
20	C5-20	C8-20	C10-20	C12-20	C15-20	C20-20
30	-	-	C10-30	-	C15-30	C20-30
40	C5-40	C8-40	C10-40	C12-40	C15-40	C20-40
50	-	-	C10-50	-	C15-50	C20-50
60	-	C8-60	C10-60	C12-60	C15-60	C20-60

. The first letter, C, in the designation stands for crack; for example, C5-20 stands for a crack with width 5 mm and depth 20 mm. The configurations of the test specimens are illustrated in Figure 2. Macrocracks were selected in this study because of the detection system and the early stage of this research. The purpose of the system was to detect significant cracks in damaged structures rapidly and widely, using UAVs in disaster situations such as post-earthquakes [22,23,24,25,26]. Currently, this system aims primarily to detect macrocracks using thermal images and machine learning; after the investigation, the microcrack test specimens will be used to improve the detection system. To manufacture the test specimens, molds were fabricated. After filling the concrete in the molds, polymer pieces of equal size to the cracks were installed at the planned crack locations with easily detached thin wax. The polymer pieces were fabricated using a 3D printer, and the strength of the concrete in the specimens was 24 MPa.

2.3. Test Setup

Thermal images were obtained from test specimens located outdoors at 1–4 pm after the specimens were directly exposed to sufficient daylight. To gather 6439 thermal images, the test was repeated over several months. The performance of a thermal camera located outdoors can be influenced by several parameters such as air temperature, humidity, illumination, cracks, and concrete surface temperature; the thermal images were therefore collected with those parameters set for machine learning. The air temperature ranged from −1.8 °C to 37.8 °C. In addition, humidity and illuminance ranged from 3.1% to 74.3%, and from 3200 lux to 91,810 lux, respectively. The test setup is shown in Figure 3, where the distance between the thermal camera and specimen was 5 m, which was determined by the minimum shooting distance of the UAV. This was because the proposed method aimed to rapidly detect the macrocrack at the unapproachable area using a UAV. The location and angle of the thermal camera towards the specimens were always fixed because the UAV with GPS can obtain ortho-thermal images with flight path planning in advance. In addition, the thermal camera was installed in the UAV instead of a normal camera with special modules. The professional thermal camera was an FLIR T530 [27], whose specifications are listed in

Table 2. Specification of the thermal camera [27].

Parameter	Specification
IR resolution	320 × 240 (76,800 pixels)
Thermal sensitivity/NETD	<40 mK, 24 °C @ 30 °C (86 °F)
Accuracy	±2 °C (±3.6 °F) or ±2% of reading
Digital camera	5 MP, with built-in LED photo/video lamp
Display	4″, 640 × 480 pixel touchscreen LCD with autorotation
Storage media	Removable SD card

. From the thermal images obtained, the raw data in the crack and surfaces were extracted at each temperature per pixel using environmental parameters. The surface temperatures at the upper and lower parts of the crack were measured because the surface conditions of concrete have more influence than the other environmental parameters. In addition, the specimen depth was decided by the previous studies [22,23,24,25,26] to avoid the rear surface effect on the crack or surface temperatures.

2.4. Test Results

An example of the thermal images obtained is shown in Figure 4. The minimum pixel size of the thermal camera was 6.55 mm when the distance between the camera and specimen was 5 m. There was no issue with measuring the cracks of the specimen in this study, where the minimum width was 10 mm, as shown in Table 1. Because the test specimen was exposed to daylight before the test, the crack temperatures were higher than the surface temperatures. Light passed through the crack and was repeatedly reflected inside the crack. In contrast, light was reflected only once at the surface of the test specimen. Therefore, crack temperatures are higher than surface temperatures. The factors affecting the difference between crack and surface temperatures are environmental parameters, such as air temperature, humidity, and illuminance during the test. An example of the three-dimensional (3D) temperature distribution at the crack is shown in Figure 5, where the differences between the crack and surface temperatures are clearly shown. The shape of the temperature distribution at the crack was similar to that of the crack. An example of raw data with the parameters is presented in

Table 3. Example of raw data.

Crack Width (mm)	Crack Depth (mm)	Crack Temperature (°C)	Surface Temperature (°C)	Air Temperature (°C)	Humidity (%)	Illuminance (lux)
10	60	11.3	10.8	15.5	23.3	7448
15	20	10.3	9.9	13.7	24.1	6686
20	10	4.8	3.8	20.4	30.6	14,967
20	40	28.8	27.2	25.1	44.4	6189

.

3. Data Analysis Result and Discussion

3.1. Parameter Information

The total number of data points was 6940, and all data were used for analysis. The statistical information and normal probability distributions of the data are shown in Figure 6 and

Table 4. Statistical information.

Features	Average	Standard Deviation	First Quartile	Second Quartile (Median)	Third Quartile	Max. Value	Min. Value
Crack temperature (°C)	24.80	13.50	16.00	26.70	37.00	47.8	−2.3
Surface temperature (°C)	24.50	13.58	15.05	26.30	36.40	47.2	−3.3
Air temperature	22.85	10.33	16.50	25.20	31.00	37.8	−1.8
Humidity (%)	37.23	12.35	27.30	40.50	45.50	74.3	3.1
Illuminance (lux)	15,630	16,040	4436	9777	22,230	91,810	3200
Crack width (mm)	15.05	8.90	10.00	15.00	20.00	60	5
Crack depth (mm)	35.31	17.22	20.00	40.00	50.00	60	10

. Standardization datasets were used to improve machine-learning performance. The crack depth was the target value for machine learning, whereas the others were input values.

3.2. Machine Learning

This study used four machine-learning algorithms (decision tree, extremely randomized tree, extreme gradient boosting, and AdaBoost), because the previous study [19] used other machine-learning techniques (multilayer perceptron, random forest, gradient boosting, and AdaBoost) and suggested that AdaBoost could efficiently predict crack depth using thermal images. To prevent overfitting, cross-validation was conducted as a resampling procedure used to evaluate machine learning on a limited dataset. k-fold cross-validation, which partitioned the dataset into two groups (training and test), was selected [28]. Machine learning used the training group to analyze data and the test group for verification. The decision tree (DT) algorithm [29] classifies the data using criteria based on data features and structures the classification into a tree shape. Decision making is provided by the classifier function, and the subgroups are repeatedly divided until they match the target data. For example, the specific classifier for performing binary classification was selected, such as where the crack temperatures of the specimen with a 20 mm crack depth were over 30 °C when the air temperatures were over 25 °C. Then, the 25 °C air temperatures could be the first classifier to estimate the crack depth. However, to prevent the overfitting and the wrong estimation flow with complicated classifiers, selection was applied. From multiple and repeated classifiers, the decision tree algorithm estimated the crack depth. As the extremely randomized tree (EXT) [30] is an advanced random forest algorithm, overfitting rarely occurs. The subsamples of the data features are randomly selected, and the algorithm trains the optimal estimators. The EXT algorithm uses the entire training group for the training estimator, whereas the random forest uses only the extracted data. To estimate the crack depth, parameters were randomly selected without information gain. For example, 25 °C air temperature was randomly selected as the feature and the optimal estimators were calculated. After that, various features were randomly selected, and the optimal estimators were calculated to estimate the crack depths. As extreme gradient boosting (XGB) [31] is developed from the gradient boosting algorithm with classification and regression trees. The training speed of XGB is much faster than gradient boosting. The main feature of XGB is the regularized loss function to prevent overfitting, which can reduce the feature size and simplify training. Therefore, XGB can be used when data and situations are complicated. For example, several decision trees, such as the mentioned example in the DT, were combined and assigned their weights. The difference between the DT and XGB, is whether using sampling data or weighted sampling data. AdaBoost (AB) [32] is similar to the gradient boosting algorithm except for the weight factors. AB grades the weight factor for the misclassification data to apply to the next decision-making iteration, which can reduce the training duration and improve the accuracy. Therefore, AB can be applied to complicated situations with large data; however, the computation cost is high. For example, the crack temperatures of the specimen with a 20 mm crack depth were over 30 °C when the air temperatures were over 25 °C. The errors of this classifier were calculated, and this weighted data affected the second classifier and this procedure was repeated to derive the best estimation.

3.3. Results and Discussions

To evaluate the performance of the machine learning, five indicators, training and test scores, ±1 and 2 mm accuracies, and mean absolute percentage error (MAPE), were calculated. The training and test scores were evaluated using the coefficients of determination (R²_train, R²_test) using Equation (1).

$$R^2=\frac{S{S}_R}{S{S}_T}=\frac{{\left({\displaystyle \sum _{i=1}^n\left(y_i-\overline{y}\right)\left(t_i-\overline{t}\right)}\right)}^2}{{\displaystyle \sum _i^n{\left(y_i-\overline{y}\right)}^2{\displaystyle \sum _i^n{\left(t_i-\overline{t}\right)}^2}}}$$

(1)

where R² is the coefficient of determination, SS_R is the sum of squares related to regression, SS_T is the total sum of squares, y_i is the real crack depth of the i-th data, t_i is the estimated crack depth of the i-th data, ${\overline{y}}_i$ is the average of the real depth, ${\overline{t}}_i$ is the average of the estimated depth, and n is the number of data.

The coefficients of determination ranged between 0 and 1. If the value approaches zero, the correlations between the real and estimated cracks are irrelevant to each other. By contrast, if the value approaches 1, the correlation between the real and estimated cracks is relevant to each other. Generally, training scores are higher than test scores; when the difference between them is large, machine learning is overfitted. Next, accuracies of ±1 and 2 mm (acc₁, acc₂) were evaluated using Equation (2). This indicates the possibility of developing microcracks.

$$acc=\frac{N_{true}}{n}$$

(2)

where acc is the accuracy, N_true is the number of estimated data points within the allowable errors, and n is the number of data points.

The mean absolute percentage error (MAPE) is a non-dimensional indicator for evaluating errors and an improved version of the root mean square error (RMSE) that does not consider size-dependent errors. MAPE can be expressed as follows:

$$MAPE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|\frac{y_i-t_i}{y_i}\right|}$$

(3)

where MAPE is the mean absolute percentage error, n is the number of data, y_i is the real crack depth of the i-th data, and t_i is the estimated crack depth of the i-th data.

MAPE should be set between 0 and 1. If the value approaches zero, the performance of the machine-learning algorithm is satisfactory. The machine-learning results for the aforementioned indicators are listed in

Table 5. Machine-learning results.

Algorithm	R2train	R2test	acc1 (%)	acc2 (%)	MAPE (%)
DT	0.9436	0.8821	76.74	80.99	7.23
EXT	0.9710	0.9606	52.33	69.83	8.27
XGB	0.9999	0.9710	87.97	92.15	2.63
AB	0.9998	0.9896	97.70	98.49	0.60

. Based on the training and test scores, all algorithms except DT can be used for detecting macrocrack depths; the test score for DT is too low to effectively predict crack depths. DT showed lower scores because it is a single learner, whereas the other algorithms had several weighted learners to predict the crack depths. In addition, the test score of AB was the highest because of the weight factor of the misclassification data for the next decision. In terms of accuracy, only the AB algorithm has the possibility of detecting microcracks. However, XGB and AB can be selected for detecting macrocracks when considering MAPE because the MAPE of the EXT was quite high and even higher than DT. The difference between XGB and AB is the random classification of XGB and weighted factor for the misclassification data of AB. Therefore, random classification should be helpful in detecting macrocrack depth because the parameters showed a complicated correlation with the crack depth, not random correlation.

4. Data Bias Analysis Result and Discussion

4.1. Data Bias Analysis

In this section, we discuss ways to prevent data bias and evaluate its effects using three bias prevention models: principal component analysis, singular value decomposition, and independent component analysis. A change in features may cause changes in other features because of covariation.

Principal component analysis (PCA) [33] is a technique used to evaluate the covariation of numerical variables, which selects smaller variable groups than the number of numerical variables. The principal component (Z_i) can be obtained by multiplying the variables (X_i) by the weight values (w_i,n) in the n-th data feature as follows:

$$Z_i=w_{i,1}{X}_1+w_{i,2}{X}_2+\cdots +w_{i,n}{X}_n$$

(4)

where Z_i is the principal component, w_i,n is the weight, X_i is the variable, and i is the number of variable groups (=1, 2, …, k, k ≤ n).

To investigate the data bias for the crack depth, six principal components were used: crack, surface and air temperatures; humidity; illuminance; and crack width. The component with the highest weight value, when considering the six principal components, was the crack temperature.

Singular value decomposition (SVD) [34] can be presented in any matrix M, which is decomposed into three matrices, as follows:

$$M=U\mathsf{\Sigma }{V}^T$$

(5)

where U is an m × m unitary matrix, Σ is an m × n diagonal matrix, V is an n × n unitary matrix, and V^T is the conjugate transpose of V.

Matrix M is a real-valued data matrix; the columns represent samples, and the rows indicate the features. This technique finds the standard axis to make an average of rows zero similar to the PCA, and the axis with the largest value is decided as the matrix U. SVD is an efficient technique for most datasets, although it focuses on variance, and it is difficult to understand strongly nonlinear data.

Because PCA [35] creates orthogonal principal component vectors by extracting the principal components, the principal components are independent of each other. However, principal components that have no correlation, in fact, would not be independent of each other. To solve this limitation of PCA, independent component analysis (ICA) was developed to classify multidimensional data into independent components.

4.2. Analysis Results and Discussions

To evaluate the data bias analysis, three techniques, PCA, SVD, and ICA, were used with six parameters. The machine-learning algorithms for each technique show five performance indicators, as presented in

Table 6. Data bias analysis.

Algorithm	Technique	R2train	R2test	acc1 (%)	acc2 (%)	MAPE (%)
DT	PCA	0.9303	0.8578	75.52	71.06	9.43
	SVD	0.9218	0.8533	74.65	71.06	10.44
	ICA	0.9229	0.8752	76.38	72.14	9.1
EXT	PCA	0.9309	0.9198	49.75	32.54	13.43
	SVD	0.9098	0.8935	44.13	24.69	14.9
	ICA	0.9304	0.9194	48.67	31.61	13.37
XGB	PCA	0.9999	0.9445	87.68	82.07	4.25
	SVD	0.9999	0.9442	84.59	78.11	5.26
	ICA	0.9999	0.9403	85.03	78.69	4.86
AB	PCA	0.9987	0.9776	90.21	82.36	3.05
	SVD	0.9097	0.8935	44.13	24.69	15.9
	ICA	0.9964	0.9765	84.38	71.71	4.32

. The value of several indicators decreased compared to the results in Table 5, whereas others remained almost unchanged. The reason for the decrease in the indicators was the data loss caused by the techniques used to classify the data groups. The performance of the SVD technique was better than that of the others for all algorithms, meaning that the correlation between crack depths and the six parameters was not random, and could be defined as simple non-linear relations. It is significant that the training scores of XGB with all techniques were excessively overestimated because their results were overfitted. However, this model has the potential to be developed for microcrack detection with accuracies up to ±1 and 2 mm, if the XGB can be controlled and prevented from overfitting. In addition, unlike DT and EXT, AB with PCA and ICA showed excellent performance in detecting macrocracks and development possibilities for detecting microcracks. PCA should be helpful for detecting microcracks in the XGB and AB. From a previous study [26] and the results of this study, AB is the best algorithm for detecting macrocracks. If overfitting can be prevented, XGB can also be used to detect macrocracks. For further studies on detecting microcracks, AB and XGB with PCA will be the best algorithms.

5. Conclusions

This study proposes a method for predicting macrocrack depths in concrete using thermal images and machine learning. Using the same test setup, 6490 thermal images were obtained over several months for artificial crack specimens with known crack widths and depths, with exposure to environmental parameters such as air temperature, humidity, and illuminance. These provided the input data for the machine learning. To detect the macrocrack depth, four machine-learning methods, DT, EXT, XGB, and AB, were selected, and data bias analysis was conducted using PCA, SVD, and ICA. Similar to a previous study, the AB algorithm was found to be the best machine-learning algorithm, with or without data bias techniques. However, XGB can also be an excellent machine-learning method if it prevents overfitting. To evaluate the development possibilities of detecting microcracks in further studies, accuracies up to ±1 and 2 mm were obtained. Based on these accuracies, AB and XGB with PCA can be used to detect microcracks. In the case of XGB, overfitting must be prevented, as in a larger test dataset. The proposed concrete crack depth prediction method using thermal images and machine learning is expected to be actively used when rapid structural safety assessments are needed. For example, if a strong earthquake occurs in a city, the structural safety of numerous buildings in the city should be rapidly investigated. In addition, the proposed method with UAVs is expected to be applied to infrastructure or buildings where human access is impossible, such as nuclear facilities and irregular or tall buildings.