RC Bridge Concrete Surface Cracks and Bug-Holes Detection Using Smartphone Images Based on Flood-Filling Noise Reduction Algorithm

Abstract

Noise reduction is a key process in digital image detection technology for concrete cracks and bug-holes. In this study, the threshold range of the flood-filling noise reduction algorithm was investigated experimentally. Surface cracks and bug-holes in RC bridge concrete were detected using mobile terminal images based on the flood-filling noise reduction algorithm. The results showed that the error range was within 10% when threshold range Θ was confined in [60, 80] as the crack width was from 0.1 mm to 2 mm. It is suitable that the threshold range Θ was selected as 70 while the measured crack width range was 0.2 mm to 2 mm. However, by reducing the values of the threshold range Θ to 50, the miscalculation was obviously eliminated. The influences of reducing values of the threshold range on bug-holes of the equivalent diameter and area were not significant. It is suitable that the threshold range Θ was elected on 50 to detect bug-holes in the concrete surface. The threshold range can be selected as a suitable value for the detection of cracks and bug-holes in order to reduce noise.

1. Introduction

The health status monitoring data of bridge structures during service is one of the important bases for enhanced bridge deterioration prediction [1,2,3], such as monitoring for structural deformation [4,5,6], concrete cracks [7,8,9,10], etc. More and more attention is being paid to the importance of bridge monitoring data-driven structure deterioration prediction in supporting cost-effective maintenance decision making [11,12,13]. The concrete surface cracks may accelerate the deterioration process of bridges, and the occurrence and severity of bridge concrete cracking serve as important indicators that maintenance is required [14,15,16]. The crack information is an important basis for adopting the appropriate rehabilitation method to fix the cracked structure and prevent any concrete structures failure [17,18,19]. On the other hand, bug-holes in the city bridge concrete surface not only affects the visual and aesthetic effect of concrete structure surface but also affects the structure durability. For structural control and health monitoring of city bridges, traditional techniques rely on technicians using specialized equipment to obtain structural response parameters, such as cracks and bug-holes in the concrete members’ surface. However, limited technicians and equipment cannot satisfy the current demand for rapid monitoring of bridge structures [20,21]. In recent years, a novel detection method with digital image detection technology for concrete cracks has been applied to engineering monitoring. The acquisition of digital images can be obtained using a fracture imager [22,23], digital camera [24,25], UAV (unmanned aerial vehicle) [26,27,28], and smart mobile phones [29,30]. It is of great importance to design or develop a simple and convenient detection method for concrete cracks to guide decisions on structural safety and reliability, such as designing a technology for smartphone or other mobile terminal-based participation in the life-cycle inspection of bridge infrastructure concrete cracks [31,32,33]. However, crack and bug-holes detection using a concrete image is well known to be difficult because of some manual traces or other reasons such as background noises of the image, irregularly illuminated conditions, shading and a blemish, and the background noises of the image, which are important interference factors [34,35]. Thus, noise reduction is a key process in digital image detection technology for concrete cracks.

Several algorithms were proposed during the image pre-processing to reduce or remove noises in the image, such as mean filtering and median filtering [36,37,38]. The method of mean filtering is based on the assumption that the spatial correlation between adjacent pixels in the image to be processed is extremely high, and that the pixel gray values are less abrupt, while noise exists independently. In this way, after the gray value of a pixel is replaced with the gray mean of adjacent pixels in its neighborhood, the purpose of smoothing the image and filtering the noise can be achieved. The median filtering needs to sort the gray values of the pixels in the neighborhood template, and then the sliding template used by the median filter generally contains an odd number of pixels. When traversing the image, the gray value of the pixels in each template is sorted from small to large, and the median value of the gray scale is taken instead of the gray value of the pixels in the center of the template. The effect of the median filter largely depends on the choice of the filter window size. A small filter window size can protect some details in the image, but the denoising effect is not notable; the filter window size has a good noise-filtering effect, but it will cause the image feature boundary to blur. So, it is a key step to look for a good noise reduction algorithm in image processing, especially for cracks and bug-holes detection.

The flood fill algorithm is another denoise algorithm. It uses the concept of water always flowing from a higher elevation to a lower one [39]. The flood fill algorithm is an algorithm to simulate the fluid diffusion process, which is widely used in the image processing process of determining connected regions in a multidimensional space [40,41,42]. In the flood fill algorithm, the thresholding method was utilized to avoid the initial seed pixels being planted manually so the denoising point could be presented [43]. The threshold range value of the flood fill algorithm has a direct impact on the noise reduction effect. There are few studies in this area, especially for the image detection of cracks and bug-holes in concrete surfaces.

So, in order to explore the suitable noise reduction algorithm for the image detection of cracks and bug-holes in concrete surfaces, an experiment on the flood-filling noise reduction algorithm was conducted in this study. Firstly, the threshold range of the flood-filling noise reduction algorithm was investigated experimentally. Then, surface cracks and bug-holes of NC bridge concrete were detected using smart terminal images based on the flood-filling noise reduction algorithm. The aim of this study was to determine the threshold range for the detection of cracks and bug-holes in order to improve the detection accuracy.

2. The Concrete Cracks and Holes Image with a Smart Terminal Camera

The performance of smart terminal camera improvements has been developing rapidly, such as the pixel resolution of smartphone postposition cameras, which started from 12 million megapixels several years ago, then developed to 13 million, 16 million, 40 million, 50 million, 60 million, 160 million megapixels, even up to 200 million megapixels now. For smartphones such as Apple, Samsung, Huawei, Oppo, Vivo, MI, etc., these brands’ phones have reached this performance level. At the same time, the pixel resolutions of mainstream tablets’ postposition cameras are more than 13 megapixels. The high-definition images of concrete surface information can be obtained by these cameras’ hardware, such as the smart terminal, and then the characteristic information of concrete surface cracks and holes can be obtained by the image processing technology (dedicated software algorithm), including areas of cracks and holes, cracks length, etc. The dedicated software algorithm can be compiled into some dedicated applications suited for smart terminals, such as smartphone, iPads, etc., and these applications can be abbreviated as APP, and to extract the cracks and holes features by dedicated applications, this process can be called app detection. In this study, the smart terminal camera equipment was parallel to the inspected concrete surface, and the shoot distance was kept at 1 m in order to reduce the influence of inclination and turning angle during shooting experiments. The results of our finished investigations which were published in the literature [29] showed that the influence law of the shooting distance had a good linear relationship under no amplification. So, the shooting distance was set as the actual distance in the influence experiment of shooting distance and could be corrected based on the linear relationship, in accordance with the literature [29].

3. Experimental Research

3.1. The Flood-Filling Noise Reduction Algorithm

In the crack image, the crack can be regarded as a closed area/or sealed area with the same gray value within a certain threshold. The difference between the gray value of the crack pixels and the surrounding background noise value can be used as an algorithm basis to reduce the background noise [39,40]. The flood fill algorithm is a kind of useful noise reduction algorithm and is useful as an algorithm to simulate the fluid diffusion process, which is widely used in the image processing process of determining connected regions in a multidimensional space [41,42,43]. It can be used as a fast algorithm to reduce noise with a difference in pixel gray values. Some initial seed pixels should be planted based on the thresholding method in the flood fill algorithm first. So, a seed pixel point in the area of the crack can be selected, as shown in Figure 1. The gray value of the surrounding target pixels can be compared with the seed pixel point based on the difference. Then, the surrounding target pixels will determine whether they belong to the filling area or not.

In order to determine whether the target pixel point belongs to the crack area or not, a task must be carried out to determine the difference in the gray values between target pixel point and the seed pixel point. If this difference is in the middle of some threshold, that is to say, the judgement was satisfied, the target point would be filled, and it would be used as the new seed point for the next round of the filling process. As shown in Figure 1, there are two algorithms: four connections and eight connections, named cross type (four-directional filling) and the Chinese character “米” type (eight-directional filling). Although the Chinese character “米” type is computational and time-consuming, it is effective in filling complex regions, and it shows strong robustness when identifying complex crack areas in crack images. In this paper, the Chinese character “米” type was used for the noise reduction algorithm during the crack target extraction.

3.2. Recursive Function of the Flood-Filling Algorithm

It is an important key problem to determine the threshold range of the difference in the gray values between the target pixel point and seed pixel point. In order to solve this problem, the recursive function of the flood-filling algorithm can be built as follows:

$$F(x,y)=\left\{\begin{array}{l}{f}_1(x,y)+{\displaystyle \sum _{i=-1}^{1}F(x+1,y+i)+F(x-1,y)}\\ +{\displaystyle \sum _{i=-1}^{1}F(x-1,y+i),I(x,y)\in [{\xi }_1,{\xi }_2]}\\ f_2(x,y),I(x,y)\notin [{\xi }_1,{\xi }_2]\end{array}\right.$$

(1)

where the function $f_1\left(x,y\right)$ indicates that the pixels at the determination $\left(x,y\right)$ are in the crack area, the gray value is filled to 0, and the function $f_2\left(x,y\right)$ indicates that the pixels at the determination $\left(x,y\right)$ are in the background area and the gray value is filled to 255.

$I\left(x,y\right)$ is the filling condition, determined by comparing the difference between the current pixel and the difference between the seed pixels in the neighborhood.

ξ₁ represents the maximum value of the negative difference between the gray value of the current pixel and the seed pixel in its neighborhood, and ξ₂ represents the maximum value of the positive difference between the current pixel and the seed pixel in its neighborhood.

3.3. The Determination of Threshold Range Experimentally

Consequently, the threshold range of [ξ₁, ξ₂] (named: [ξ₁, ξ₂] = Θ) are determined through some experiments in this paper. A standard crack detection card was selected, which passed the measurement certification as the detection object (it is shown in Figure 2). The length, width, and area of the characteristic regular cracks in the crack detection card are all known. In the experiment, the different threshold ranges were set to monitor the standard cracks with APP to investigate the influence of the threshold range Θ, such as [−100, 100], [−90, 90] … [−50, 50].

4. The Experimental Results Discussion

4.1. Threshold Range Θ

The results of the threshold range experiment are shown in Figure 3. The results showed that there was no suitable threshold range when the crack width was 0.05 mm, 0.1 mm, or 3 mm, and they indicate that the flood-filling algorithm was not applicable. The error range was within 10% when the threshold range Θ was confined in [60, 80]. So, it was reasonable and optimal that the mid-value of threshold range Θ was selected as 70 while the measured crack width range was 0.2 mm to 2 mm.

4.2. Regular Crack Detection Using the Flood-Filling Algorithm

In order to inspect the flood-filling algorithm of crack detection under the threshold range Θ = [60, 80] (the mid-value of threshold range Θ is 70), the measurement accuracy experiments of different crack widths were conducted under different detection distances and different zoom ratios. The detection distances were 0.5 m, 1 m, 1.5 m, and 2 m, and the camera zoom times were 1, 3, 5, 7, and 9, respectively. The regular crack images were photo-taken using a standard crack detection card with a smartphone (type: Huawei Mate-20 Pro). The results from the authors’ studies (the literature from [29,30]) showed that the accuracy of APP detection was high when the detection distance was within 2 m for cracks with widths from 1 mm to 3 mm. The relative measurement error was generally less than 10%, and the measurement accuracy of a high zoom ratio was higher than 95%. However, the APP detection conditions were limited for cracks; the reason was that some fine cracks less than 1 mm wide could not be detected with the increase in detection distance. In order to ensure accuracy and measurement, a high zoom ratio should be used within a close detection distance. The suggested detection distances are shown in

Table 1. The applicable ranges of the APP detection method and recommended zoom factor.

Crack Widths Range (mm)	The Recommended Zoom Ratio in Different Distance (Times)
	0~0.5 (m)	0.5~1 (m)	1~1.5 (m)	1.5~2 (m)
0~0.1	≥9	-	-	-
0.1~0.5	≥5	≥7	≥9	-
0.5~1	≥3	≥3	≥5	≥7
1~3	≥3	≥3	≥5	≥5

Note: “-”: represents that the APP detection method is not recommended within this test range.

based on the threshold range Θ = [60, 80]. That was the applicable range of the APP detection method for different crack widths of (0~3) mm, and the recommended zoom ratio at different detection distances are shown in Table 1 [29,30].

4.3. The Graphical Distinction Between Bug-Holes and Cracks

The feature of the graphical distinction between bug-holes and cracks is obvious; that is, the cracks on the concrete surface are narrow and long. The bug-holes tend to be round, although the shape of the bug-holes is irregular. So, a sharp characteristic coefficient λ can be used to distinguish the bug-holes and the cracks, and the coefficient λ can be obtained by Formula (2).

$$\mathit{\lambda }={\displaystyle \frac{L^2}{A}}$$

(2)

where L is the perimeter of shape and A is the area of shape, unit is pixel. In special cases, for positive circles, λ = 4π ≈ 12.56, unit is pixel. Some partial representative shape coefficients of holes and cracks are shown in Figure 4. The shape coefficient was applied to filter the image to successfully remove the cracks and eliminate the interference of the cracks when detecting the holes. The filtering effect is shown in Figure 5.

The perimeter and the area of the feature profile can be obtained by calculating the number of pixels in the contour periphery or the number of pixels in the contour whole. The calculation formula is as follows:

$$L=\delta \times L_{pix}$$

(3)

$$S={\delta }^2\times S_{pix}$$

(4)

where δ is the size of individual pixel point, $L_{pix}$ is the number of pixels of the feature edge profile, unit is pixel, and $S_{pix}$ is the number of all pixels within the feature region, the unit of S and $S_{pix}$ is pixel square.

Three parameters can be used to evaluate the characteristic value of concrete surface bug-holes, such as the equivalent diameter $\overline{D}$, the area rate $A_r$, and the pixels quantity of bug-holes.

$$\overline{D}=\sqrt{{\displaystyle \frac{4S}{\pi }}}$$

(5)

$$A_r={\displaystyle \frac{S}{S_c}}$$

(6)

where $S_C$ is the concrete surface area in the image, pixel square

5. RC Bridge Concrete Surface Cracks and Bug-Holes Detection

5.1. Engineering Background and Engineering Application

In the regular inspection of normal concrete bridges, the cracks and holes detection in the concrete surface of bridge pier structures is one of the important aspects of health monitoring. In order to test the application effect of the bug-holes detection APP which was designed and developed by our research team in actual engineering, the bug-holes detection field investigation on the cap beam of a ramp bridge pier in Wenzhou city was conducted. The detection scene is shown in Figure 6.

5.2. The Measurement of Concrete Cracks

According to the recommended values of threshold range Θ, the four cracks (named crack №①, …, crack №④) were measured and analyzed using APP. The diagrams of the concrete crack extraction effect are shown in Figure 7. The crack extraction effect results indicated that the recommended values of threshold range Θ were suitable. The crack №④ did not reflect the real situation because of concrete scaling, and the measurement values are not included in

Table 2. Measurement results of concrete crack length and area.

№	Crack Length/mm	Crack Area/mm2
①	111.003	80.992
②	285.916	204.367
③	160.163	105.194

. The concrete crack length and area results of crack №①, …, crack №③ are shown in

Table 3. Measurement results of concrete crack width.

Location	APP /mm	Fracture Width Measuring Instrument /mm	Absolute Error /mm	Fractional Error /%
A	0.712	0.76	0.048	6.316
B	1.167	1.08	0.087	8.056
C	0.583	0.57	0.013	2.281
D	0.394	0.37	0.024	6.486
E	0.646	0.61	0.036	5.902

. The concrete crack width measurement results in locations A, B, C, D, E, F are shown in Table 3 and compared with the values using a fracture width measuring instrument (Type: HC-CK102, accuracy of reading: 0.01 mm, detection distance: 0–8 mm, magnification: 40). The concrete crack width measurement results showed that the largest absolute error was 0.087 mm, the largest fractional error was 8.056%. It indicated that the recommended values of threshold range Θ were suitable and the accuracy of APP detection method was excellent.

5.3. RC Bridge Concrete Surface Bug-Holes Detection with the Cracks Filtered

According to the recommended values of threshold range Θ, the bug-holes detected would be miscalculated, such as Θ = 70. While reducing the values of threshold range Θ to 50, the miscalculation was obviously eliminated, and the regarding influences of reducing the values of threshold range on bug-holes, the equivalent diameter and the area were not significant. So, the threshold range Θ was elected on 50. Figure 8 shows that two bug-holes were detected in picture of Crack №①, the value of the equivalent diameter $\overline{D_0}$of № 0 bug-hole detected was 2.53 mm, and the value of the equivalent diameter $\overline{D_2}$ of № 1 bug-hole was 2.10 mm. The area of № 0 bug-hole was 5.01 mm², and the area of № 1 bug-hole was 3.48 mm².

Similarly, the rust spots were identified as having holes while values of threshold range Θ ≥ 65 in Figure 9, and while Θ = 50, this miscalculation can be eliminated.

5.4. Classification and Detection of the RC Bridge Concrete Surface Bug-Holes Equivalent Size

The distribution characteristics of bug-holes include equivalent pore size, equivalent bug-hole area, area ratio, the number of bug-holes of specific equivalent pore size, bug-hole location, etc., as shown in Figure 10. So, the bug-holes size $\overline{D}$ of the concrete surface can be divided into the following five grades: ≤1 mm, 1~3 mm, 3~5 mm, 5~8 mm, and ≥8 mm. The results of classification and detection of concrete surface bug-holes are shown in

Table 4. The classification and detection results of concrete surface bug-holes in Figure 10.

Bug-hole size $\overline{D}$/mm	1.62	2.34	3.25	2.68	2.19	1.74	0.97
Bug-hole quantity	0.137	0.252	0.242	0.383	0.256	0.116	0.060
Area ratio/%	55	47	25	67	27	5	11

, and it shows that the characteristics of the APP output hole are very clear.

6. Summary and Conclusions

6.1. Summary

Noise reduction is a key process in digital image detection technology for concrete cracks and bug-holes. The mean filtering, the median filtering, and the flood fill algorithm are three denoise algorithms. Obviously, the mean filtering and the median filtering of a digital image is confirmed, while the pixel gray value threshold of the flood fill algorithm is variable. A different threshold range could be selected as a suitable value for the detection of cracks and bug-holes in order to reduce noise. So, the noise reduction effect of the generic flood filling algorithm is better and with more use scenarios. The threshold range for the detection of concrete cracks and bug-holes was proposed in this study. However, the law of threshold selection still needs to be further explored in subsequent studies.

6.2. Conclusions

In this paper, the threshold range Θ of the flood fill algorithm which was used in the image processing was determined via experiments. The main conclusions are as follows:

(1)

The threshold range Θ could be a suitable value for the detection of cracks and bug-holes in order to reduce noise.

(2)

The error range was within 10% when threshold range Θ was confined in [60, 80] as the crack width was from 0.1 mm or 2 mm. It was suitable that the threshold range Θ was selected as 70 while the measured crack width range was 0.2 mm to 2 mm.

(3)

Reducing the values of threshold range Θ to 50, the miscalculation was obviously eliminated, and the influences of reducing the values of threshold range on bug-holes the equivalent diameter and the area were not significant. It was suitable that the threshold range Θ was elected on 50 to detect bug-holes in concrete surfaces.