Image-Based Crack Detection Methods: A Review

Abstract

Annually, millions of dollars are spent to carry out defect detection in key infrastructure including roads, bridges, and buildings. The aftermath of natural disasters like floods and earthquakes leads to severe damage to the urban infrastructure. Maintenance operations that follow for the damaged infrastructure often involve a visual inspection and assessment of their state to ensure their functional and physical integrity. Such damage may appear in the form of minor or major cracks, which gradually spread, leading to ultimate collapse or destruction of the structure. Crack detection is a very laborious task if performed via manual visual inspection. Many infrastructure elements need to be checked regularly and it is therefore not feasible as it will require significant human resources. This may also result in cases where cracks go undetected. A need, therefore, exists for performing automatic defect detection in infrastructure to ensure its effectiveness and reliability. Using image processing techniques, the captured or scanned images of the infrastructure parts can be analyzed to identify any possible defects. Apart from image processing, machine learning methods are being increasingly applied to ensure better performance outcomes and robustness in crack detection. This paper provides a review of image-based crack detection techniques which implement image processing and/or machine learning. A total of 30 research articles have been collected for the review which is published in top tier journals and conferences in the past decade. A comprehensive analysis and comparison of these methods are performed to highlight the most promising automated approaches for crack detection.

1. Introduction

Annually, millions of dollars are spent to acquire various tools and assets to carry out defect detection from key infrastructure which includes roads, bridges, buildings, and water bodies [1]. Civil structures such as roads, bridges, buildings, and pavements are often exposed to extreme physical stress which may be caused by natural disasters like earthquakes, catastrophic incidents like blasts or daily usage. Such incidents can either cause a complete collapse of the structure or may lead to physical damage that is often represented in the form of cracks. Usually, cracks emerge at a microscopic level on the surface of the infrastructure component [2]. These cracks make the component weak, reduce its loading capacity and lead to discontinuities on the surface [3,4,5]. If such cracks are detected at an early stage, further damage can be reduced [6]. Undetected cracks can however spread through the surface and may lead to the complete collapse of the structure, resulting in fatalities, injuries, and financial loss. Manual methods of crack detection involve experts who examine the component visually and the use of specific tools to identify any deficiency in the component [6]. However, this method is tedious, labour extensive and prone to human error. Automatic crack detection deals with using technologies to identify cracks from infrastructures. The level of degradation can be determined by analyzing the length, width, depth and severity of a crack. These measures can be used to make decisions regarding the classification of the crack, durability of the structure and its usage [7]. Using the traditional inspection procedures which involve manual inspection, it is very time-consuming to determine the crack measures which make it difficult to make inference regarding the level of degradation. Hence, for a quick, effective, and reliable damage assessment, the crack detection process must be automated to replace the manual defect inspection methods. Some testing methods like laser, infrared, thermal, radiographic, and thermal testing approaches have been used in the past to automate the process of crack detection [8,9,10]. However, more recently, there has been an increasing trend of using image-based methods for detecting cracks. These methods involve capturing images of the target component and analyzing them programmatically to find and classify cracks. Such methods are fast, less expensive, and robust. The methods can be categorized into two types namely as image processing and machine learning. The image processing methods do not require a model training process and involve the use of filters, morphological analysis, statistical methods, and percolation techniques for the detection of crack [11,12]. On the other hand, the machine learning process involves the collection of a dataset of images, which are supplied to the selected machine learning model for training. Such methods may involve image processing steps for preprocessing and noise removal, but the crack detection task is done by the trained machine learning model [13].

Figure 1 shows the basic architecture of an image processing-based method for crack detection. First using a camera or any other imaging mechanism, high-resolution images of the target component are collected. The images are then preprocessed which involves using filters, segmentation and other approaches to remove noise and shadows from the image. The image may be converted to grayscale or binary form if required by the specific crack detection method being used. The resultant image is applied to the crack detection procedure which uses image processing techniques like edge detection, segmentation, or pixel analysis to highlight or segment the cracked part in the image [14]. Parameter estimation involves calculating the specific properties of the detected crack such as its length, width, depth and density. Such measures help in making decisions regarding the severity of a crack.

The basic steps to build a machine learning model for crack detection are depicted in Figure 2. In the first step, a dataset must be collected showing surface cracks, which are to be detected using the machine learning model. Previously, a study led by Lin et al. used 30,000 low-resolution images for training [15]. The images are preprocessed using image processing techniques to reduce noise, remove shadows and adjust other properties such as size and brightness of images. The cracks in these images then undergo pixel-wise annotation or labelling, where the defected pixels are annotated in the image. This step can be performed manually or using a labelling tool. One such example of labelling is to set crack pixels as white or “1” in the image while the remaining pixels will be set as black or “0”. After this step, a machine learning model needs to be selected, which is to be used for crack detection. In past studies various machine learning models such as support vector machines (SVM), CNN, and decision trees have been used for crack detection [16]. A cost/loss optimization function is then formulated to minimize the loss or cost of training the model. A weighted cross-entropy loss function can be used for this purpose [17,18,19,20]. The designed model will then be trained using the set of annotated images collected in the dataset. After training the model, a new set of images will be applied to the model to see if the model successfully classifies the cracked regions in the image.

In this paper, we present a review of the image processing and machine learning-based methods for crack detection which have been proposed over the last decade. For this purpose, 30 research articles from journals and top tier conferences were retrieved and the respective crack detection methods were analyzed concerning the crack detection technique proposed, its features, performance, dataset details and the specific component to which the method is applicable. The results of each method are documented along with the corresponding limitations. A comparative analysis of these methods is conducted to highlight the most promising methods for automatic crack detection. The paper is organized as follows: Section 2 presents the methodology followed to collect data for the study. The article retrieval and screening process are explained in detail in this section. Section 3 presents the result of the study, discussing the crack detection techniques proposed in the articles and highlighting the strengths and limitations of each method. Section 4 discusses the results, analyzing the features of the methods and presenting a general picture of the current advancement in this domain. Section 5 summarizes the goals, outcomes, and achievements of this research along with the prospects.

2. Materials and Methods

The aim was to assess the development in this field and how these advanced tools are facilitating post-disaster scenarios. To achieve the desired goals, top journals were searched for recent and significant work carried out in the domain. The review process was carried out in two phases i.e., retrieving articles and screening them.

To retrieve the research articles for this study, the chosen search engines were Scopus, Google Scholar, Science Direct, Elsevier, and Springer for finding the latest developments and interdisciplinary research in the field. The next step was to formulate a set of queries to be used in each of the search engines to retrieve the articles. The major aim was to fully exhaust the search database and retrieve a maximum number of articles matching the domain of interest. We used three categories of terms representing the subdomains, to extract a variety of research articles. After entering the search queries, a set of articles ranked based on their relevance were retrieved. The first category of phrases was formulated to retrieve articles that proposed flood prediction models using image processing technologies that utilised multispectral sensors. The phrases were formed by using keywords related to flood prediction which include “flood prediction”, “flood risk analysis” and “flood hazard mapping” along with phrases like “image processing” and “artificial intelligence”. The second category of terms was formulated to retrieve articles that proposed flood prediction methods using these technologies. For this purpose, we used flood prediction keywords along with the keyword “edge detection”, “mining patterns from images, Synthetic Aperture Radar, and “Image-based flood Alarm model”.

After the first phase based on article retrieval, the articles were passed through a screening phase to further narrow down the selection criteria. Four assessment criteria were defined to evaluate the articles:

(1)

No duplicates

(2)

Time interval: 2010–2021

(3)

Document type: research article, abstract, book chapter

(4)

English language only

Thus, by filtering the articles based on these metrics, the most recent, applicable, and unique research articles written in the English language were extracted. From the 1250 articles retrieved in the first phase, 94 articles passed all four selection criteria. Hence, this review is based on these screened articles. The number of articles from each term category i.e., image processing, artificial intelligence and integrated approach that passed the screening phase is shown in Figure 1. The articles were screened for duplicates, non-English articles, and review papers. Around 520 papers were removed for duplicates, 240 for non-English articles and 396 for review papers. Hence, overall, 94 papers were finally collected as an output of the screening phase.

In this section, the method adopted to retrieve the data relevant to the study is presented. The article retrieval and screening process are discussed in detail. Figure 3 shows the overall methodology followed for the study. To retrieve the articles most relevant to the research questions proposed in the study, we define two categories of research articles. These are:

• Cat-1 Crack Detection using Machine Learning
• Cat-2: Crack Detection using Image Processing

After defining the categories, we defined keywords related to each category. Figure 4 shows the keywords are search phrases that are to be entered in the search engines of a research article repository. The aim was to generate a maximum number of keywords for each category to retrieve as many relevant research articles as possible. For this purpose, we defined a basic set of keywords which are: {crack detection using machine learning, crack detection using image processing, crack detection, crack measurement, crack classification}. A keyword search process was conducted to find the most frequent keywords reported in the literature, relevant to the previous set of keywords. The retrieved keywords include: “segmentation”, “support vector machine”,” technology”, ”computer vision” and “classifier” as shown in Figure 4. The retrieved keywords illustrate that data that is to be gathered for the research must revolve around “crack detection technologies”. Keywords from this set were used in combination with the base set of keywords, to completely exhaust the database and retrieve a maximum number of articles relevant to the study.

The next step was to specify the resource from where the research articles are to be retrieved. For this purpose, recent research articles from a wide range of journals and conferences were retrieved. Search results were obtained for each specific search phrase entered on the search engine. Screening criteria were selected based on the time frame relevance to the topic of study, and authenticity of websites for article retrieval.

More precisely, the following assessment criteria for screening were outlined:

(1)

Published between 2010 to 2020

(2)

English Language only

(3)

Websites must be: MDPI, Elsevier, IEEE Xplore, Arxiv, Science Direct

(4)

Article type must be research article, review or book chapter (letters, abstracts and comments are excluded)

(5)

No duplicates

The articles meeting the specified criteria for screening were downloaded. A total of 107 articles were retrieved at this stage. After this step, each article was carefully studied so that only relevant articles were kept for the research. This involved reading the abstract, methodology and results of each article. After the detailed content analysis, the articles not found relevant to the defined categories were discarded. At the end of this process, a total of 30 research articles were finally selected for the study. These articles met all the assessment criteria and were found relevant to the research questions posed in this study.

Figure 5 shows the year-wise distribution of the research articles which were selected for the study. The graph shows a growing trend among studies that are moving towards machine learning methods for crack detection, especially during the most recent years (2016–2020). The use of image processing methods was common at the start of the decade (2010), however, with the rapid advancement in the field of artificial intelligence (AI), the machine learning methods took over and gained rapid attention of various researchers who aimed to introduce automation in the crack detection process.

3. Results

A total of 30 research papers were collected because of the article retrieval and screening method. Figure 6 shows the overall distribution of these articles in the image processing and machine learning domains. The pie chart shows that 67% of articles proposed crack detection methods that employed machine learning models while 33% used image processing methods. In this section, a comprehensive review of these papers is presented. Comparative analysis of these techniques is done in tabular forms. The performance outcomes, technique, dataset, imaging method and limitations of each method is presented to make inference regarding the feasibility of the method and its applicability in real-time crack detection tasks.

3.1. Image Processing Based Crack Detection

Table 1. Image Processing Methods for Crack Detection.

Method	Features	Domain	Image Details	Imaging Device/Source	Results	Limitations	Ref.
Recursive Tree edge pruning	Crack Detection	Pavement	206 images 800 × 600	-	Precision = 0.79 Recall = 0.92 F-Measure = 0.85	Increased runtime (up to 30 s)	[4]
GP and Image Filtering	Crack Detection	Concrete	17 (varying resolution)	Digital Camera	Accuracy = 80%	-	[10]
Gabor Filter	Crack Detection	Pavement	5 336 × 339 pixels	Canon IXUS 80 IS	Precision up to 95%	Results presented on 5 images only	[13]
Particle Filter	Crack Detection & Measurement	Civil Structures	14 12 MP	IP Camera	Error Range = 7.51–8.59%	-	[14]
Beamlet Transform	Crack Detection, measurement & Classification	Pavement	256 × 256 pixels	-	A method is fast & robust to noise	Cant calculate crack width; manual setting of thresholds prevents full automation	[18]
Median filter, Hessian Matrix, probabilistic relaxation	Crack detection	Noisy Concrete Surfaces	60 images 640 × 480 pixels	SONY Cyber-shot DSC-F828	AUC = 0.9903	-	[21]
FPHBN	Crack Detection	Pavement	500 + 1969 + 206 + 118 + 38 (varying resolution)	Crack500, GAPs384, CrackTree200, CFD, Aigle-RN	AIU = 0.081 Time = 0.241 s/image	Method is not real time	[22]
Canny edge detector, dilate operators, Frangi filter	Crack Detection	Bridges	72 images 4288 × 2848 resolution	UAV	Detection rate = 98.7%	-	[23]
UAS Operator	Crack Detection and measurement	Bridges	Real-time crack detection	DJI Mavic Pro	DJI Mavic Pro most suitable camera to visualize cracks	UAS not stable in the absence of GPS and windy atmosphere	[24]
Shi-Tomasi feature point detection	Crack Detection	Bridges	Real-time crack detection	consumer-grade digital camera	The system is robust to varying illumination conditions and complex textures	Accuracy affected by noise-limited camera resolution	[25]

summarizes the image processing-based methods reviewed in this paper for crack detection. Each of the methods has been discussed in the subsequent sections.

3.1.1. Tree Structures

One of the many challenges faced by researchers in crack detection is the presence of noise in the images, which makes it difficult to identify crack pixels [4,9,22,23]. To tackle this problem, Zou et al. presented an approach to reduce shadows from the input pavement images to make the cracks more prominent and easier to identify. A tensor voting scheme was proposed that builds probability maps for cracks which involve using visual clues of vicinity and connectivity [4]. Minimum spanning trees (MSTs) were built which show all the possible links of the determined crack sources. The unwanted edges were eliminated to get the output crack curves. The dataset for validation of the model consisted of 206 pavement images. The system takes up to 16 s to determine cracks from an input image. The precision-recall and F-measure values have been recorded as 0.79, 0.92 and 0.85 respectively indicating high accuracy and significance of the methods.

3.1.2. Genetic Programming

Nishikawa et al. employed image processing methods for automatic crack detection from concrete images. Major cracks are detected through genetic programming (GP) [26]. Genetic programming has been used previously for space optimisation by implementing augmented Lagrangian genetic algorithm, optimize largescale structures by application of the fuzzy genetic algorithm, controlled system problems by introducing floating-point genetic algorithm, applied for management purposes for senor allocation and forecasting accident durations [11,12,13,27]. The remaining noise from the images is reduced by applying filters. Minor cracks are then detected by repeatedly applying an image filter to the areas around major cracks. Spatial derivatives of the intensity patterns in the crack segments are calculated to determine crack widths. An accuracy of 80% has been achieved on a dataset of 18 test images.

3.1.3. Image Filters

Some concrete materials have more textured surfaces, which makes the crack detection task even more challenging. To deal with this issue, Salman et al., proposed an image processing-based approach to detect cracks from pavement images. A Gabor filter was employed for multidirectional crack detection [13]. A Gabor filter is a linear filter that analyzes the texture in a region to determine the presence of content having a particular frequency in a specific direction. Hence, this method is found to be very effective to detect cracks from pavements having rich textures. The experimental results showed a precision of 95%.

Statistical methods have been applied to images to detect cracks. Lins and Givigi applied a statistical filtering approach to detect cracks and then used image-processing based methods to determine the lengths and widths of the detected cracks. A particle filter is used for crack detection on civil structures [14]. This filter is originally designed to monitor objects in clutter. A vector is used to represent the position of each object at time t. The error range recorded by this method is between 7.51% to 8.59%. To determine the width and length of each crack, the total number of crack pixels determined by the crack detection method is multiplied by the pixel resolution. The angle of a crack is calculated by drawing a line between any two points on the crack and then using trigonometric rules.

Fujita and Hamamoto worked on a crack detection system that is robust to the presence of noise in concrete images. The test image first passes through a pre-processing stage where a median filter is applied to reduce noise and shade present in the image [28]. To make the cracks more prominent in images, a line filter is applied which is a multi-scale filter that uses a Hessian matrix. Multiscale Hessian filtering is useful for the enhancement and segmentation of narrow fractures in 3D image data. Finally, to detect cracks, a probabilistic relaxation method is applied to the resultant image. Adaptive thresholds are applied to further improve the precision of crack detection. The method is tested on 60 concrete images containing noise. The area under the curve (AUC) recorded by the system is 0.9903, indicating the accuracy and precision of the system.

Yeum and Dyke worked on detecting defects from bridge images. They particularly focused on detecting cracks that are present close to the bolts on steel surfaces. Images are captured from varying angles and positions [29]. From each image, the region of interest (ROI) is the area near to bolt but not the bolt itself. Median filter, canny edge detector and dilate operators are used to extracting bolts from each image. After removing bolts from the images, a Hessian matrix-based method called Frangi filter is applied to detect crack like edges. The detection rate of the system is 98.7%, depicting the efficiency of crack detection using this method.

3.1.4. Beamlet Transform

Ying and Salari proposed a beamlet transform method for the detection and classification of cracks on pavement images. Beamlets are an organization of line segments at varying angles, scales, and locations [30]. This method is used to retrieve linear features from images such as edges and lines. Hence, it is deemed as effective in detecting cracks, which are curvilinear features, from the noisy and textured surface images of pavements. The extracted crack parts are connected, and each crack is then categorized into one of the four categories, which are: (1) horizontal, (2) vertical, (3) transversal and (4) block.

3.1.5. Unmanned Aerial System (UAS)-Based Approach

Dorafshan et al. worked on crack detection from bridge images. A UAS was utilized and the impact of lighting and distance of a crack from the camera on crack detection results was investigated. Hence a measure called “achievable crack to platform (ACP) distance” was evaluated [31]. This measure represents the maximum distance of the camera from the platform from where a crack can be accurately detected. The UAS operator monitored the image and adjusted lighting and distance until the crack was visible. This operator also measures the length and width of each crack. Among all the cameras tested by the system, the DJI Mavic Pro was the best camera used in the UAS for enhancing crack visibility [32,33,34,35].

3.1.6. Shi-Tomasi Algorithm

Kong and Li worked on detecting cracks from steel bridges. A short video is captured from the target bridge structure. Differential features resulting from the opening and closing of cracks are tracked at each frame of the video [36]. Feature detection was done using the Shi-Tomasi algorithm. The gaps in the surface movement of the bridge part in the video stream were identified and tracked to detect cracks. Experimental results showed the robustness and efficiency of this method even in the presence of varying illumination conditions. One limitation is the increased dependency on the camera resolution, as the accuracy was reduced on low-resolution videos [37,38,39].

3.2. Machine Learning-Based Crack Detection

Table 2. Machine Learning-based Crack Detection.

Method	Features	Domain	Dataset	Device/Source	Results	Limitations	Ref.
GoogleNet CNN, FPN	Crack delineation	Civil Structures	64000 crack & 64000 non-crack images	Canon Camera	Precision = 80.13% Recall = 86.09% F-Measure = 81.55%	Need 16 s to find cracks on an image of 6000 × 4000 pixels	[40]
CNN	Defect detection	Calf Leather	584 images 400 × 400 pixels	Robotic Arm	Accuracy = 91.5% (training), 70.35% (testing)	-	[41]
CNN	Crack detection	Pavement	500 images 3264 × 2448	Smartphone sensor	Precision = 0.8696 Recall = 0.9251 F-Measure = 0.8965	-	[42]
FCN	Crack Detection and density evaluation	Concrete	20,000 crack & 20,000 non crack 227 × 227	Public Dataset	AP = 89.3% F-Measure = 89.3%	Reduced performance for crack density evaluation in the presence of noise	[43]
K-means clustering, Gaussian Models	Crack detection, measurement and Characterization and severity assessment	Road	84 images 1536 × 2048 pixels	Digital Camera	F-Measure = 97%	Less accuracy in detection of narrow cracks (<2 mm)	[44]
STRUM, SVM, Adaboost, Random Forest	Crack Detection and density evaluation	Bridge	100 images 1920 × 1280 pixels	Robotic Scanning	Accuracy = 95%	-	[45]
SVM, MDNMS	Crack Detection	Road	7250 images 4000 × 1000 pixels	Line scan cameras, laser and HW-SW	Precision = 98.29% Recall = 93.86%	-	[46]
CNN	Crack Detection	Pavement	260 training images 512 × 512 pixels	CrackTree, CRKWH100, CrackLS315, Stone331	F-Measure = 0.87	Does not work well for cracks on stone images	[47]
CNN	Crack Detection	Pavement	500 3264 × 2448 pixels	Smartphones	Accuracy = 91.3%	Results subject to location variance	[48]
FCN	Crack Detection & Measurement	Pavement & Walls	800 (varying resolution)	Digital Camera	Accuracy = 97.96%	-	[49]
Random Structured Forests, SVM	Crack Detection & Characterization	Road	38 + 118 images 480 × 320 pixels	CDN, AigleRN Datasets	Precision = 96.73%	Crack width not measured; Not tested on videos	[50]
MorphLink C, ANN	Crack Detection & Characterization	Road	100 0.99 mm per pixel	LRIS	MSE = 0.0094–0.0105	-	[51]
NB-CNN	Crack Detection	Nuclear Power Plant Components	147344 crack, 149460 non-crack 120 × 120 pixels	20 captured videos 720 × 540 pixels	Average AUC= 96.8%	To avoid overfitting a large number of training images required; Reliance on GPU	[52]
Morphologic Image Processing, Logistic Regression	Crack Detection	Steel Slabs	644 + 323 images 0.1 × 0.1 × 0.0053 mm (width length depth) resolution	3D Profile Data	Accuracy above 80%	-	[53]
Transfer Learning (CNN)	Crack & Sealed Crack Detection	Pavement	800 images 2000 × 4000 pixels	ImageNet Dataset	recall= 0.951; precision= 0.847	-	[54]
Canny Algorithm, decision tree heuristic	Crack Detection & classification	Pavement	400 images 320 × 320 pixels	Digital Camera	Success rate= 88% for crack detection 80% for crack classification	Not tested in real-time	[55]
Morphological analysis, segmentation, extreme learning machine classifier	Crack Detection & Classification	Subway Tunnels	38000 images 6144 × 1024	CMOS line scan cameras	Accuracy > 90%	Parameters setting need to be done for images of different resolutions	[56]
CNN	Crack Detection	Pavement	2000 3D images 1 mm resolution	Image Library of 5000 3D images	Precision (90.13%), Recall (87.63%) and F-measure (88.86%)	Reduced accuracy in finding hairline cracks	[57]
Morphological operations, NN, SVM	Crack Detection, depth perception	Civil Structures	1910 non- crack, 3961 crack images 5184 × 3456 pixels.	Canon EOS 7D	NN: Accuracy = 79.5% SVM: Accuracy = 78.3%	-	[58]
Deep convolutional encoder-decoder network	Crack Detection	Road	527 images	Black-box Camera	Recall = 71.98% Precision = 77.68% Intersection of Union = 59.65%	-	[59]

summarizes the machine learning methods reviewed. Each of the methods has been discussed in the subsequent sections.

3.2.1. Convolutional Neural Network (CNN)

The CNN model is commonly found in the literature for crack detection. This model has three layers of neurons: convolutional layer, pooling layer and fully connected layer. The convolutional layer extract features from images which enables it to learn to distinguish between crack and non-crack image. The pooling layer is used for down-sampling the image and reducing its size by adjusting its dimensions. A fully connected layer is used in the final stage of a CNN model, as it takes the output from the previous layer as input and maps it to an output label.

Crack delineation involves both identification and segmentation of cracks from images. Ni et al., automated these tasks using feature map fusion and pixel classification [40]. A CNN architecture named GoogleNet CNN was applied for crack classification. The output was processed using feature pyramid network (FPN), which contains fusion layers and consecutive convolutional layers which collectively perform crack delineation. The results showed that the network was able to do crack delineation accurately achieving a precision of 80.13%. One limitation is the long processing time of the network as it takes 16 s approximately to detect all the cracks from an image having a resolution of 6000 × 4000 pixels. Deep learning with CNN architecture for surface crack detection with enhanced computation. The proposed model was compared with other methods such as VGG16, Inception, and ResNet. The proposed shallow CNN architecture model achieved maximum accuracy of 99.8% in the minimum computation.

To assure driving safety, detecting defects from pavements and roads have also been a subject of interest for many researchers. A defect detection system has been proposed that detects cracks from pavement images using a CNN model [41]. A total of 500 images for the dataset have been collected using a smartphone sensor. The CNN model is trained using these images along with manually annotated ground truth data. To speed up the training process a graphics processing unit (GPU) is utilized. In CNN architecture, rectified linear units (ReLU) is used as an activation function, to further accelerate the process. The model recorded a precision, recall and F-measure of 0.8696, 0.9251 and 0.8965 respectively.

Zou et al. [4] proposed DeepCrack, which is a trainable deep CNN model that detects crack from input pavement images, by utilizing high-level features such as ridges to represent cracks. The DeepCrack network is constructed on the architecture of SegNet which is based on encoder-decoder design. The features produced in the convolutional stages of the encoder network and the decoder network are fused in a pairwise manner. Four standard crack datasets are used for evaluations which are: CrackTree, CRKWH100, CrackLS315, Stone331. From these datasets, three are used for testing and one for training. Experimental results show an F-measure above 0.87. The method works well for pavement images and did not show good results on input stone images.

Regular examination of the components in nuclear power plants is deemed necessary to ensure the safety of the workers. A total of 20 videos were captured from components in a nuclear power plant and integrated CNN and a naïve Bayesian (NB) classifier to fuse and analyze the data acquired from each video frame. In this framework, CNN detects cracks from each input frame while NB eliminates the false positives in the output [42]. The data fusion approach preserves each crack’s spatiotemporal consistency. This system achieved a hit rate of 98.3%.

CrackNet is a CNN-based model presented by Zhang et al. [3] for crack detection on pavements. There are no pooling layers in the CrackNet architecture, unlike the traditional CNN model. This architecture ensures accuracy up to the pixel level as the image length and width remains unchanged in all layers. Convolutional and fully connected layers are the hidden layers of this network [26]. Training of the model was done using 1800 three-dimensional (3D) images. A total of 200 3D pavement images were captured for testing the model. The system achieved a precision, recall and F–measure of 90.13%, 87.63% and 88.86%.

Yang et al. [15] used a variation of CNN called fully convolutional network (FCN) for pixel-level segmentation of cracks on the images of walls and pavements. The FCN model is trained using multiple kinds of crack images [15]. Pixel skeletons having a width of only one pixel are used to represent the crack segments predicted by the FCN model. The morphological features of these cracks, like length, width and topology are then measured using these skeletons. For validation, the crack segments acquired by the model are compared to ground truth and the results produced by FCN model crack detection systems. The overall accuracy achieved by the crack segmentation system is 97.96%. This method outperforms the CrackNet model in that it provides pixel-level segmentation and reduces the training time. However, its performance is less than CrackNet in terms of accuracy.

Bang et al. [31] worked on the detection of road cracks at the pixel level. A deep convolutional encoder-decoder network was used for this purpose [31]. The encoder part of this network consists of convolutional layers which extract the crack features while the decoder part consists of de-convolutional layers which find the crack locations in images. The model was trained using 427 black box images which were extracted from black-box videos and was tested on 100 images. The system recorded a precision, recall and intersection of a union of 77.68%, 71.98% and 59.65% respectively.

Pauly et al. [14] used CNN to detect cracks from pavement images . A total of 500 images were collected using smartphones from pavements across the United States of America (USA). The resolution of each image was 3264 × 2448. Each image was then partitioned into patches of 99 × 99 pixels. Next, these patches were labelled as either cracked or non-cracked. The experimental results showed that by increasing the depth of the neural network, better performance outcomes are achieved. A limitation of this method is the location variance problem according to which, when the model is tested on pavement images collected from a different location, the performance is reduced. An accuracy of 91.3% was recorded by this crack detection system.

Researchers have proposed several methods to calculate the dimensions and measurements of the detected cracks such as length, width and depth along with the densities. Dung and Anh have proposed a system that detects cracks from concrete images and calculates their densities. They performed semantic segmentation of cracks on concrete images [5]. The system uses an encoder-decoder FCN model. A total of 500 annotated images were used to train the encoder based on VCG16. The average precision achieved by the network is 90%. The system detects cracks and also calculates the densities of these cracks with reasonable performance. To measure the density of a crack, the total number of segmented crack pixels is divided by the total number of pixels in the image. This ratio is referred to as the pixel density [40,41,42,43].

Differentiating between cracks and sealed cracks is another challenge that exists in the literature. Zhang et al. [23] worked on crack detection on pavement images having complex textures while focusing on the problem of distinguishing between cracks and sealed cracks having identical width and brightness. A CNN model is trained for the classification of a pavement image into crack/sealed crack and background sections [43]. A section-wise thresholding process is applied to the output image for pixel-based segmentation of cracks and sealed cracks. A curve identification method based on tensor voting is employed to extract the crack or sealed crack. A total of 800 images are used to test the system. The system showed a recall of 0.951 and a precision of 0.847.

3.2.2. K-Means Clustering

Crack characterization methods deal with classifying the detected cracks based on their types [44,45,46]. Oliveira and Correia presented such a crack detection and classification approach, that does not require manual labelling of dataset images [6]. A total of 84 road images were captured for the training of the system using a digital camera. Unsupervised training of the system is done using images from the training dataset. A K-means clustering method and a mixture of two Gaussian models were tested to detect cracks from the input images. The results showed that the Gaussian models’ mixture showed the best F-Measure i.e., 93.5% and the least error rate that is 0.6%. In the case of recall, this method achieved the second-best performance which is 95.5%. The detected cracks are classified either as longitudinal cracks, transversal cracks or miscellaneous. This is done by analyzing the connected components of each crack and computing a crack skeleton. Using the crack skeleton, a crack width is determined. The width is further analyzed to determine the crack severity level. One limitation is the lack of accuracy of the system in the detection of narrow cracks (width < 2 mm).

3.2.3. Logistic Regression

In steel products, consistent casting causes lasting cracks on the surfaces of steel slabs [47,48,49]. Hence, to avoid wasting time and costs in dealing with defective products, it is crucial to detect these cracks at an early stage. Landstrom and Thurley presented a crack detection and measurement system that uses morphological image processing. Initially, 80% length of a crack in an image is extracted by applying segmentation and minor defects or cracks are discarded [22]. After that, statistical classification is performed using logistic regression on these segmented images which detects all the major cracks. Overall accuracy recorded by the system is above 80%.

3.2.4. Feature Pyramid and Hierarchical Boosting Network (FPHBN)

Yang et al. [15] proposed FPHBN to detect cracks on pavement surfaces. To assess the performance of crack detection on images, the researchers presented a new measurement method called “average intersection over union” (AIU) [15]. The method was evaluated on five standard crack datasets. The AIU achieved by the method was 0.081 and the time taken to show output on a single image is 0.241 s. Other studies have used this technique for pavement and concrete crack detection [50,51,52,53].

3.2.5. Support Vector Machines (SVM)

Gavilan et al. [8] presented a road distress detection method. Road images were captured by a vehicle having line scan cameras, laser beams and the required Hardware and Software (HW-SW) for scanning and storage [8]. After preprocessing of images, a multiple directional non-minimum suppression (MDNMS) method is applied for crack detection. A linear SVM classifier is used to differentiate between various pavements across Spain to determine optimal parameters for crack detection. The performance of the crack detection method is improved by adapting parameters specific to the pavement. The method achieved a precision of 98.29% and recall of 93.86%. Several studies have applied SVM for crack detection [54,55,56].

3.2.6. SVM and Random Forest

Defect detection from bridges play a vital role in the maintenance of their structure and ensuring the safety of people who use them. Prasanna et al. performed crack detection on bridge images captured using robotic imaging. They proposed a crack detection method called spatially tuned robust multi-feature (STRUM). Machine learning methods: SVM, Adaboost and random forest are used to finally classify the crack and non-crack pixels [7]. Robust curve fitting is utilized to locate the possible crack regions, despite the existence of noise in the images. An accuracy of 95% is achieved using the STRUM classification approach as opposed to a 69% accuracy which has been achieved using a traditional image processing approach. A continuous bridge mosaic is created using series of images from the robot. This mosaic is further used to develop crack density maps [57,58,59].

3.2.7. SVM and Artificial Neural Network (NN)

A crack detection system for civil structures that performs segmentation by considering the depth parameters has been proposed. Crack depths are determined by performing reconstruction of 3D scenes [27]. A distinguishing factor of this method is that it retrieves the whole crack from the input image while the other methods that use edge detection just segment the cracked part. Morphological methods are adopted for crack segmentation and extraction. NN and SVM classifiers were used for final crack classification. For the NN model, the accuracy was 79.5% while for SVM, 78.3% accuracy was recorded. Similar work has been reported using SVM and NN for enhanced precision for crack detection [60,61,62].

3.2.8. Artificial Neural Network (ANN)

The fragmentation of cracks in the images is a problem frequently faced by researchers. To tackle this issue, Wu et al. [19] proposed a crack defragmentation method called “MorphLink-C”. This technique connects different crack parts present in an image [16]. A dilation transform is applied to group crack fragments. These fragments are then connected by applying a thinning transform. Crack width is also determined using this approach. The system is developed specifically for crack detection in road and pavement images. An artificial neural network (ANN) is used to perform the final classification to label the images as “crack” or “no-crack”. The MorphLink-C method has been demonstrated to improve the classification accuracy while reducing the training time of the classifier [63,64,65].

3.2.9. Random Structured Forests

CrackForest is a crack detection system for roads [66,67,68], such as Shi et al. [15] which uses random structured forests. This model targets the issue of inhomogeneity of the intensity of cracks present in the road images. To detect cracks in such images, integral channel features are applied to achieve an improved depiction of cracks [15]. After this step, a random structured forests method is employed to detect cracks. This method can accurately detect arbitrary and complex cracks from images. An SVM model is employed to classify the cracks concerning their type. Instead of having fixed labels for the crack type, this method extends the crack types to thousands of dimensions. This enables the system to characterize the arbitrary cracks as having complicated forms. The overall precision achieved for crack classification is 96.73%.

3.2.10. Decision Tree

Fernandez et al. used various image processing and machine learning methods to detect cracks from 400 pavement images. The operations include morphology-based filter, canny edge detector, bilateral filter and transformation using logarithms [23]. Finally, a decision tree classifier was used to classify the crack according to its type. The crack types are transverse cracks, longitudinal cracks and alligator cracks. The success rate of crack detection was 88% and for crack classification, the success rate was 80%. Application of a decision tree approach has been useful for crack detection [69].

4. Analysis and Discussion

In this section, we present an analysis of the crack detection studies reviewed in this paper based on the functions they have covered, the source of the dataset, the domain to which the method is applicable and the performance of the method.

4.1. Functionality Based Analysis

All the reviewed articles proposed crack detection methods. However, a few of them also focused on crack classification, which deals with categorizing the crack according to its type e.g., transversal or longitudinal cracks, and sealed cracks. Many articles performed measurements of the detected crack. For example, various methods were proposed to measure the crack length, width, depth and density [34,40,41,42]. Hence, crack classification and measurement are some important problems that have been tackled in several studies giving them an edge over studies that just focused on the crack detection task. Figure 7 shows that most studies (16) focused on crack detection only, while 10 of them dealt with both crack detection and measurement. Just four studies proposed methods for classifying cracks after detecting them. Hence, this analysis points towards the need for further development in the classification methods for cracks. Figure 7 presents the distribution of the articles based on their features or the functionality they have provided.

4.2. Crack Classification Analysis

Different articles were reviewed in the study focused on crack classification.

Table 3. Studies focusing on crack classification.

Crack Types	Ref.
Longitudinal cracks, transversal cracks or miscellaneous.	[6]
Types based on dimensions	[15]
Crack, sealed crack	[20]
Transverse cracks, longitudinal cracks and alligator cracks.	[23]

shows the crack types identified by researchers while categorization of the detected crack. This table illustrates that all the articles proposed some unique categories to classify cracks. Longitudinal (caused due to late or shallow saw cutting, inadequate base support, and built-in warping and curling) and transversal cracks (extends along the cross-section of structure, usually perpendicular to its centerline) have been tackled in two studies [6,23] while a new crack type called “sealed crack” was proposed in [20]. The authors of [15] classified the cracks based on their dimensions, hence concluded that thousands of types exist for the cracks detected in their study.

4.3. Crack Measurement Analysis

Ten articles reviewed in this study proposed mechanisms to measure various properties of the detected cracks. These properties include the length, depth, width and density, etc. of the detected crack. Such measures give important inference about the crack severity and the condition of the structure of the component from where it is detected.

Table 4. Studies focusing on Crack Measurement.

Crack Measurement	Ref.
Length, width, depth, density	[5]
Density	[7]
Widths	[10]
Length, width & topology	[12]
Angle, width & length	[14]
Depth	[27]
Width, severity	[6]
Width	[16]
Length, width	[24]
Length, width	[29]

shows that the majority of research articles focused on finding the length and width of the crack. Other measures such as density, topology and angle severity have also been calculated in some of the articles. Among these methods, the method proposed in the article [5] calculates the highest number of measures, followed by [12] and [14] which introduce two new measures which are topology and angle, respectively.

4.4. Image Source

Before developing any image-based crack detection system, collecting images for the dataset is another problem that needs to be resolved. From the analysis conducted on the reviewed articles, we can infer that 70% of researchers build their dataset (Figure 8). This involved using various types of cameras such as line cameras, robotic scanning, and smartphones. Developing custom datasets is preferred by researchers as this enables them to capture the specific requirements of the crack detection algorithm being used. About 30% of researchers used pre-built crack datasets, which contained crack images. These datasets are being used as standards for testing and training crack detection models. Examples of these datasets include Crack500, GAPs384, CrackTree200 and CFD, Aigle-RN. Such datasets are widely used in machine learning models, as they require a large number of images or training, hence, it is deemed more feasible to use standard domain-specific datasets.

4.5. Domain of Crack Detection

Each method reviewed in this study is designed to detect cracks from a specific element or component. In this review, it is observed that a large majority of papers focused on detecting cracks from pavements (Figure 9). A large portion of articles focused on civil structures in general. Hence, these methods can detect cracks from any infrastructure element like roads, bridges, walls and pavements. Crack detection from other materials such as leather, steel and nuclear power plant components are also considered in three of the studies [2,17,19]. However, the rest of the articles only focused on infrastructure components like subway tunnels, roads, bridges, pavements and walls. Hence, this points towards the need to focus on developing methods for crack detection which can be applied to other elements as well such as leather, steel, textiles and other industrial products.

4.6. Precision Level

The applicability and reliability of any crack detection method can be assessed by reviewing its performance outcomes. In the reviewed articles, researchers have used varying performance measures to validate their systems such as accuracy, AUC, precision, recall and F-measure. However, precision was used most commonly among these articles for the evaluation of the crack detection model. The

Table 5. Performance (precision) analysis of the various crack detection methods.

Precision (%)	Ref.
95–100	[8,15]
90–95	[13,26]
85–90	[3,5]
80–85	[1,20]
75–80	[4,28]

shows the precision values of these methods. This shows the methods presented in [8,15] recorded the highest precision values which ranged between 95% to 100%. All of these methods showed reasonable performance as shown by the precision values which ranged between 75% to 100%.

5. Gaps and Challenges

Crack detection is a challenging task as cracks have an irregular form with no specific shape or size. Hence, no pre-built method can be used to recognize a crack or distinguish it from the background texture and noise. In this paper, we reviewed crack detection methods belonging to two domains, namely image processing and machine learning. By examining the methodology presented in the articles and the limitations, we can infer that both these domains pose specific challenges when it comes to accurate crack detection from any component. Crack classification has been rarely addressed in the articles. The type of the detected crack facilitates the inspection process by giving important indications regarding the nature of the crack, its cause and its severity. More research needs to be focused on developing methods to classify the detected crack so that the system can recognize the type of the crack and the maintenance operations can be focused on dealing with the specific type of crack. Image processing methods have given satisfactory performance on the custom datasets built by the researchers. However, these methods are dependent on the lighting conditions, resolution of the images and the level of the noise present in the images [12,34,41]. Also, the surfaces of the concrete structures have varying textures as they are exposed to external disturbances and may not have the same texture even if they are built from the same material. Hence, they may not give as good results when a new image having a different texture, brightness, resolution or noise level is given as input. Furthermore, crack detection in transversal direction is not enough accurate as compared to measurement carried out longitudinally. this difference in directional measurement could be an issue when establishing a relationship between the width and longitude of the crack. Hence, the practical applicability of using image processing-based methods is still obscure [10,24,35].

On the other hand, machine learning methods also pose several limitations to the researchers. Increased processing time has been observed in many methods [70]. Many methods require manual parameter setting of the model which limits the full automation of the crack detection method [24]. Reliance on GPU is another limitation as some methods do not work as efficiently without it due to a large number of images in the dataset [17]. To avoid overfitting of the model it becomes necessary to train the model using a large dataset. These methods require extensive labelling of data images. In practical scenarios, a limited option for labelling is also available, so the acquisition of labels can be a difficult task [26]. Due to the difference in surface conditions, a different algorithm may be needed to accurately detect the cracks. In addition, crack detection is carried out offline so performance in real-time detection is poor. Therefore, there is a need to improve the performance of algorithms and detection accuracy in real-time. The method selected should be robust as factors such as climatic conditions may impact crack detection. Limitations in terms of the need for large datasets in order to train neural network can be overcome by empirically decomposing the fitted networks into ensembles of low-bias sub-networks, thus making use of small data sets. In addition, deep learning methods can be applied to unsupervised tasks, using a small dataset that does not require extensive labelling of data, thus reducing time and cost [48,50].

The presence of noise, shadows, blemishes and other disturbances in the images is a problem commonly faced by researchers in using both image processing and machine learning methods [5,30]. Hence, more research needs to be conducted to develop methods that can remove noise and other irregularities from images [23,71].

6. Conclusions

The paper focuses on the domains of image processing and machine learning for crack detection. It reviewed state-of-the-art crack detection methods that have been developed in the past decade with results are published in top-tier journals and conferences. A total of 30 research articles were reviewed that were screened after applying criteria and performing a detailed examination of their content. These articles have been assessed based on the method they have used, the dataset details, imaging method, performance outcomes, features and limitations. From the analysis, it can be inferred that a wide range of articles focuses on crack detection only. However, calculating the dimensions of the crack was not performed in the majority of the studies. Measurements of the detected crack such as its length, width, density and depth give important indications regarding the state of the component and its durability and thus helps in making decisions regarding the structure’s further usage. More research needs to be focused on crack measurement and the system must be able to provide a final verdict regarding the severity of the detected crack using different techniques and algorithms. Most methods showed excellent performance outcomes as the precision values for crack detection ranged between 75% to 100%. Another observation is that the researchers prefer using their custom-built dataset, which caters to the specific requirements of the method used in the system. Most of the reviewed crack detection techniques apply to civil infrastructure and concrete components. Overall and specifically in recent years (2016 to 2020), most of the studies have focused on using machine learning methods instead of image processing for crack detection. Among these methods, CNN has been most frequently used for this problem. In the future, the research can be extended to include assessment criteria to assess the performance such as the runtime of the algorithm, its resource consumption and applicability in real-time scenarios.