Skip to Content
You are currently on the new version of our website. Access the old version .
SensorsSensors
  • Article
  • Open Access

6 November 2024

15 Pages

CrackNet: A Hybrid Model for Crack Segmentation with Dynamic Loss Function

,
,
,
,
and
1
National Engineering Research Center of Communications and Networking, Nanjing University of Posts & Telecommunications, Nanjing 210003, China
2
Artificial Intelligence of Key Laboratory of Sichuan Province, Sichuan University of Science and Engineering, Yibin 644000, China
3
College of Harbour, Coastal and Offshore Engineering, Hohai University, Nanjing 210098, China
*
Author to whom correspondence should be addressed.
This article belongs to the Section Intelligent Sensors

Abstract

Cracks are a common form of damage in infrastructure, posing significant risks to both personal safety and property. Along with the development of deep learning, visual-based crack automatic detection has been widely studied. However, this task is still challenging due to complex crack topology, noisy backgrounds, unbalanced categories, etc. To address these challenges, this research proposes a novel hybrid network, named CrackNet, which leverages the strengths of both CNN and transformer. On the encoder side, CNNs are employed to extract multi-level local features, while transformers are used to model global dependencies. Additionally, a strip pooling module is introduced to suppress irrelevant regions and enhance the network’s ability to segment narrow and elongated cracks. On the decoder side, an attention-based skip connection strategy and a mixed up-sampling module are implemented to restore detailed information. Furthermore, a joint learning loss combining Dice and cross-entropy with dynamic weighting is proposed to mitigate the effects of severe class imbalance. CrackNet is trained and evaluated on three public crack datasets, and experimental results show that the proposed model outperforms several well-known deep neural networks, with a particularly noticeable improvement in recall rate.

1. Introduction

Crack detection has become increasingly critical in recent years due to heightened concerns regarding infrastructure safety and maintenance. During both construction and long-term operation, the formation of cracks is often accelerated by material deterioration and environmental factors such as temperature fluctuations and heavy traffic. If these surface defects remain undetected, they can compromise structural integrity and pose significant safety risks [1]. Traditional manual inspection methods typically involve sketching and documenting cracks by hand. However, this approach is labor-intensive, time-consuming, and subject to environmental influences, making it challenging to ensure the reliability of results. Additionally, certain areas, such as the undersides of bridges, are difficult to access through manual inspection. Recent advancements in computer vision and machine learning, present effective solutions for automating crack detection [2]. These technologies enable timely and accurate evaluations, thereby enhancing maintenance practices and extending the lifespan of critical infrastructure.
Given the importance and complexity of this problem, extensive research has been conducted over the past few decades. Recently, deep learning (DL)-based crack detection methods have demonstrated superior performance [3,4,5,6,7] and can be broadly classified into three categories [2] within the field of computer vision, as shown in Figure 1. The crack classification model needs to determine whether an image contains a crack or not. Despite being an image-level binary classification task, it is still challenging due to the presence of other surface defects that may appear similar to cracks. The purpose of crack detection is achieved using bounding boxes to automatically identify the presence and location of cracks in images. There are usually two types of methods for crack detection: one is the two-stage method represented by Fast R-CNN [8], and the other is the one-stage method represented by YOLO [9]. Crack segmentation refers to the process of separating and extracting pixel-wise crack regions from other parts in an image, which can be used to obtain the quantification of parameters, such as crack length and width of cracks. Therefore, crack segmentation models are preferred for their ability to provide the most valuable information for crack risk level assessment. These advantages make crack segmentation the current deep learning trend in automated crack detection.
Figure 1. Crack Detection Tasks.
However, crack segmentation is an extremely challenging problem. Firstly, cracks are widely distributed on the surfaces of various components of civil infrastructure, such as bridges, dams, and roads, which can lead to issues like intensity inhomogeneity, low contrast, and cluttered backgrounds in crack images. Additionally, cracks are typically elongated and narrow, yet topologically complex. This not only makes it difficult to segment fine cracks but also results in class imbalance, causing the model to be biased toward segmenting the background. This bias is reflected in segmentation results that have lower recall rates. Moreover, the widespread distribution of cracks and many inaccessible scenes make it challenging to collect sufficient crack image data. While unmanned aerial vehicles have partially alleviated this issue, there remains a need to maximize the use of the currently limited dataset for groundbreaking work.
Motivated by the discussions above, we propose a novel model named CrackNet for crack segmentation. This model combines convolutional neural networks and transformer blocks. The convolutional module helps to extract local detail features of the cracks, aiding in the segmentation of fine cracks, while the transformer leverages long-range dependencies to address the complex topological structures of the cracks. The proposed method consists of three modules: Encoder Module (EM), Decoder Module (DM), and Attention Module (AM). Specifically, to deal with the class imbalance issue, a dynamic weight loss is introduced.
The main contributions of this paper can be summarized as follows:
  • CrackNet proposes a novel architecture in which: (1) The encoder structure is based on an improved hybrid model of CNN and transformer, enabling the establishment of remote dependencies at a high level while retaining local feature extraction capabilities; (2) A novel spatial attention module is added to each skip connection. This module emphasizes the overlapping regions of two feature maps and supplements fine-grained features, effectively reducing background interference; (3) We propose using strip pooling instead of traditional square pooling. This adaptation is specifically tailored to the elongated nature of cracks, enhancing the model’s ability to detect fine, narrow features while minimizing interference from irrelevant background noise.
  • We introduce a dynamic loss function to address the issue of imbalanced data distribution, in which the loss weights are dynamically updated based on the model’s different learning stages. This adaptive approach not only enhances the model’s performance by improving recall rates for underrepresented classes, but also reduces the likelihood of overfitting to the dominant classes. Furthermore, the proposed weight updating mechanism can be easily extended to other application scenarios, making it a versatile solution for various tasks involving imbalanced datasets.
  • The proposed method has been validated on multiple public crack datasets, demonstrating overall performance superior to that of other models.
The rest of this paper is organized as follows. In Section 2, we review research related to the proposed method. In Section 3, we provide an overview of the proposed network structure and loss function for refined crack detection. Section 4 presents implementation details, evaluation metrics, experimental results, and visualization analysis. Section 5 concludes the paper.

3. Methods

3.1. Overview of the Proposed Method

In this paper, CrackNet is proposed for crack segmentation, which consists of an encoder and a decoder (outlined in Figure 2). The encoder mainly includes CNN module, transformer module, and strip pooling module. CNN module is used to extract multi-scale feature maps. The transformer module is used to explore global contextual information to help the model locate cracks in complex backgrounds, while the strip pooling module is primarily designed to address the problem of background interference caused by unrelated cracks in traditional square-kernel convolutions. The decoder is mainly composed of multiple cascaded up-sampling modules. Specifically, this paper uses a hybrid up-sampling method (combining nearest up-sampling and bilinear interpolation up-sampling), and adopts attention modules to better utilize the information carried by the skip connections.
Figure 2. Overview of the proposed CrackNet.

3.2. Hybrid Encoder

In the encoder end, a hybrid model is adopted that integrates convolutional and transformer mechanisms to effectively capture the long-range interactions across different segments of a crack image, leveraging both local and global insights. This methodology comprises three key components: CNN-Module, Trans-Module, and strip pooling. These elements are strategically employed to investigate and understand the intricate connections among diverse portions of the crack.
(1)
CNN-Module
As shown in Figure 2, the CNN module consists of several convolutional layers, which generate feature maps at different resolutions. Each layer is composed of two consecutive convolutional blocks followed by a max-pooling layer with a step size of 2 to generate next resolution feature map. The process of the CNN-module can be formulated as follows:
F e 1 = ConvBlock ( X ) F e l = ConvBlock ( MaxPool ( F e l − 1 ) ) ,   l = 2 , ⋯ L − 1 F e L = ConvBlock ( F e L − 1 )   ,
where X represents the input image, and F e l denotes the feature map of the l − t h layer. The L − t h feature is subsequently input into the Trans-Module, and other layer’s feature maps are linked to decoder through skip connections.
(2)
Trans-Module
The architecture of the Trans-Module is depicted in Figure 3. Here, we take the convolutional feature map at its lowest resolution as the input, denoted as F i n p u t ∈ ℝ H × W × C , where H × W specifies the dimensions of the input feature map, and C indicates the number of channels. The process begins by dividing the input into N patches of size P × P , and subsequently reshaping each patch into a 1-dimensional vector. These patches are then flattened F p i ∈ ℝ P 2 ⋅ C | i = 1 , ⋅ ⋅ ⋅ , N and subjected to a linear transformation to project them into D-dimensional space. To preserve the spatial information of these patches, this article selects a learnable position encoding to generate the final visual tokens, as described below:
z 0 = F p 1 E ; F p 2 E ; ⋅ ⋅ ⋅ ; F p N E + E p o s ,
where E ∈ ℝ P 2 · C × D represents the trainable linear mapping, E p o s = ℝ N × D denotes the position embedding. z 0 serves as the input to the transformer block, which is composed of L blocks of multi-head attention (MSA), and multi-layer perceptron (MLP). The process of transformation can be expressed as follows:
z l ′ = MSA ( LN ( z l − 1 ) ) + z l − 1 ,   l = 1 ⋅ ⋅ ⋅ L
z l = MLP ( LN ( z l ′ ) ) + z l ′ ,   l = 1 ⋅ ⋅ ⋅ L
where L N denotes the layer normalization operator.
Figure 3. Transformer Module.

3.3. Bottleneck Layer

Cracks are typically characterized by their narrow features. However, when utilizing standard pooling methods, there is an inevitable inclusion of numerous irrelevant areas, especially when dealing with objects of irregular shape. This makes it obvious that standard pooling techniques fall short in effectively processing crack patterns. Consequently, strip pooling [42] is introduced to extract features specifically from the horizontal and vertical dimensions.
The strip pooling strategy, as depicted in Figure 4, involves several key steps. First, the output feature of the transformer block, denoted as z l , is reshaped into the same dimension as F e L . Next, it is compressed along the horizontal and vertical directions using average pooling, resulting in two new feature maps F v ∈ ℝ H × 1 × C and F h ∈ ℝ 1 × W × C . Following this, 1D convolution with a kernel size of 3 is applied to expand the pooled maps back to their original size, resulting in the expanded feature maps F e v ∈ ℝ H × W × C and F e h ∈ ℝ H × W × C . These expanded feature maps are then summed pixel-wise. Finally, after undergoing 1 × 1 convolution and sigmoid processing, the resulting feature map is multiplied with the original input feature map at corresponding pixels to obtain the ultimate output result F o u t s ∈ ℝ H × W × C . The entire process can be formulated as follows:
F o u t s = σ Conv 1 × 1 F e x ⊕ F e h ⊗ F i n p u t s ,
where σ ⋅ refers to sigmoid function, ⊕ denotes element-wise add, and ⊗ is element-wise multiplication.
Figure 4. Strip pooling module.

3.4. Attention Decoder Module

The attention-driven decoder plays a crucial role in reconstructing the crack region and is composed of the mixed up-sampling block, attention block, and convolutional block. At each stage of the decoder, a sequential operation involving mixed up-sampling and convolution is applied. To effectively leverage the encoder features obtained via skip connections, an attention module is integrated into the decoder architecture. This comprehensive design ensures that the reconstruction process is guided by attention mechanisms that enhance the utilization of relevant information from the encoder, ultimately contributing to the accurate reconstruction of the crack region.
Given the output feature of the decoder F d l - 1   ∈   ℝ H / 2 × W / 2 × C , it undergoes parallel up-sampling using linear and nearest neighbor methods, resulting in F ^ l − u p l ∈ ℝ H × W × C and F ⌢ n − u p l ∈ ℝ H × W × C , respectively. These two feature maps are then merged to obtain the final up-sampling feature map F u p l = F ^ l − u p l ⊕ F ^ n − u p l .
To achieve a more precise segmentation of cracks, a novel spatial attention module (Figure 5) is proposed and appended to each skip connection, which emphasizes the overlapping regions of two feature maps but also supplements fine-grained features, effectively reducing background interference. Given the encoder feature F e l ∈ ℝ H × W × C and the up-sampling feature F u p l ∈ ℝ H × W × C , they are first passed through a 1 × 1 convolution layer, with channel dimension reduced from H × W × C to H × W × C / 2 . And then an element-wise addition is applied to generate the fused feature map F ^ f l ∈ ℝ H × W × C / 2 . Subsequently, the attention feature map W s p a ∈ ℝ H × W is activated by ReLu, followed by a 1 × 1 convolution layer and Sigmod activation.
W s p a = Sigmod Conv 1 × 1 ReLu ( F ^ ) .
Figure 5. Attention block.
The output of the attention module is calculated as follows:
F e a l = W s p a l ⊗ F e l .
Furthermore, the lth layer feature map of the decoder is computed as follows:
F d l = ConBlock ( Cocat ( F e a l , F u p l ) ) .
Finally, the feature map passed through a 1 × 1 convolution layer and a sigmoid layer to obtain the segmentation results.

3.5. Dynamic Loss Function

The task of crack detection differs from others due to its small and elongated characteristics. This causes an issue of extremely unbalanced categories in the image.
Hence, a mixed loss function that combines adaptive weighted cross-entropy (WCE) [37] and Dice [38] is proposed to alleviate this issue,
L = L D i c e + L w C E
The calculation method for L D i c e and L w C E is as follows:
L D i c e = 1 − 2 ∑ i N y i y ^ i ∑ i N y i 2 + ∑ i N y ^ i 2
L w C E = − ∑ i N w y i ⋅ log y ^ i + ( 1 − w ) ( 1 − y i ) ⋅ log ( 1 − y ^ i )
where y i represents the real label and y ^ i represents the posterior possibility of each individual pixel within a ground-truth or prediction map that has a total number of N pixels. w denotes the weight, which directly affects the final result.
Since the model is updating during the training process, using a constant weight is not suitable. In the early learning process, the model is far from accurate and with a steep initial decline in the loss function, so too large w may increase the false detection rate in the crack area. When the decline of the value of the loss function slows down, the greater weight will further improve the problem of category imbalance and increase the recall rate of crack areas. Based on this intuition, a dynamic weight updating mechanism is proposed, as detailed in Algorithm 1.
Algorithm 1. Dynamic cross-entropy weight updating.
1. Input: The maximum numbers of iterations N m a x and epochs N e p o c h ; update interval N interval = α ⋅ N max / N e p o c h ; mean loss in one interval L m e a n , initialized by the last N interval iterations; the number of loss decreases M L _ d o w n ; the number of loss increases M L _ u p ; initial values are 0.
Output: The updated weight parameters w
2. For  n i t e r   <   N max  do
3. Compute L n according to Equation (9)
4. if  L n < L m e a n      //
5.  M L _ d o w n = M L _ d o w n + 1 ; // the number of loss decreases
6. else if ( L n > L m e a n )
7.  M L _ u p = M L _ u p + 1 ;    // the number of loss increases
8. End if
9. if  M L _ u p + M L _ u p =   =   N interval // the number of iterations meets the interval.
10. if M L _ d o w n − M L _ u p > T h r   //
11.  w = w + ( 1 -   N i t e r N max ) // increase weight
12. else w = w − ( 1 -   n i t e r N max ) // decrease weight
13. end if
14.  M L _ d o w n = 0 , M L _ u p = 0 ; update L m e a n ;
15. end if
16.  n i t e r   =   n i t e r   + 1
17. End for
The underlying idea behind weight adjustment is to assess, within an interval, whether the model’s loss after each iteration, compared to the average loss from the previous cycle, has increased or decreased. If the number of decreases exceeds the number of increases, the weight assigned to the crack category is further increased; conversely, it is decreased. For the robustness of the method, the weight is only increased when the difference between the number of decreases and increases exceeds a certain threshold T h r . Practically, the weight is updated at set intervals for robustness. The initial weight value w i n i t i a l can be selected based on personal experience, or foreground–background ratio, etc.

4. Experimental Results and Discussion

The proposed CrackNet was evaluated on three public crack datasets, including DeepCrack [11], Crack500 [12], and CFD [43]. Moreover, we also conduct comprehensive ablation analysis on the effect of the proposed model and the dynamic weight updating mechanism based on the CFD dataset.

4.1. Datasets and Preprocessing

(1)
DeepCrack: The dataset is composed of 537 RGB color images with manually annotated segmentations, which are of a fixed size of 544 × 384 pixels. The dataset was divided into a training set with 300 images and a testing set with 237 images.
(2)
Crack500: This dataset contains 500 images and corresponding pixel-level annotations of size around 2000 × 1500 pixels, all taken by mobile phone. The dataset is divided into 250 images of training data, 50 images of validation data, and 200 images for testing.
(3)
CFD: This dataset consists of 118 images of size 480 × 320, which was taken by an iPhone 5 on an urban road surface in Beijing, China. All images were annotated at pixel level. This dataset is very challenging due to the complex background, including shadows, oil spots, water stains, and lane lines. We divide 60% and 40% of this dataset into training and testing, respectively.

4.2. Data Augmentation

To increase the diversity of the data, we argument the datasets by random clipping, flipping, and rotation operations. Hence, the DeepCrack and Crack500 datasets were augmented by 4 times, while the CFD dataset was augmented by 2 times. Furthermore, Bilateral filter and Gaussian filter are adopted to eliminate noise interference. Due to the large image size and limited memory of GPU, each image is resized to 224 × 224 pixels. It is worth noting that the proportion of crack pixels in each database is less than 1%, which helps to examine the efficiency of the proposed approach for imbalance class segmentation.

4.3. Parameters

All the experiments mentioned in this paper were based on the Ubuntu 16.04.6 LTS system in NVIDIA Tesla P100 GPU with 16 G memory. Moreover, to control variables, each experiment was set to 60 epochs, a learning rate of 0.01, and the same learning rate decay strategy.

4.4. Performance Metrics

In experiments, the F1-score, the intersection over union (IoU), precision (Pr), recall (Re), and accuracy (Acc) are selected to evaluate the performance of the model.
P r = T P T P + F P
R e = T P T P + F N
F 1 = 2 × P r × R e P r + R e
I o U = T P T P + F P + F N
A c c = T P + T N T P + T N + F P + F N
where TP, FP, and FN refer to the true positive, false positive, and false negative, respectively.

4.5. Results Comparisons

To illustrate the performance of the proposed model, we select DeepCrack [11], UNet [15], TransUNet [32], and Swin-Unet [33] to compare with ours on three public datasets. All experiments were conducted using open-source implementation, based on the principle of controlling variables, and all model parameter settings are kept consistent.
(1)
Performance on the DeepCrack Dataset
The results on the DeepCrack Dataset are listed in Table 1. Obviously, the proposed CrackNet achieves the best results in four metrics, including F1, IoU, Re, and ACC. Although higher Pr could be achieved by TransUNet, our model obtained better tradeoff between precision and recall.
Table 1. Performance on the DeepCrack Dataset.
(2)
Performance on the Crack500 Dataset:
The results on the Crack500 Dataset are listed in Table 2. Due to the high complexity of this dataset, our proposed model only has a 1% improvement in F1, but it has a significant effect on recall which it turns out solves the category imbalance to some extent.
Table 2. Performance on the Crack500.
(3)
Performance on the CFD Dataset:
On the small-scale dataset CFD, the model we proposed still performs well thanks to data argumentation and the combination with CNN and transformer. Therefore, complementary integration of CNN and transformer is feasible. As can be seen from Table 3, the comparison models have significant shortcomings in recall due to category imbalance. In contrast, the CrackNet network proposed in this work shows a noticeable improvement in recall rate. From an application perspective, this would eliminate crack threats as soon as possible.
Table 3. Performance on the CFD.

4.6. Qualitative Results Comparisons

To provide a more intuitive comparison of the segmentation results, this subsection visualizes the segmentation results of some sample images.
(1)
Qualitative results on the DeepCrack
As shown in Figure 6, it can be seen that the proposed model is particularly sensitive to narrow and elongated cracks, especificallythose located in cluttered backgrounds. For instance, in the first-row sample, where the background includes pedestrian crosswalk markings, other models (especially the DeepCrack) erroneously identify the ground markings as cracks (red box). In the fourth image, the contrast and brightness of the image are relatively low, which poses a challenge for crack segmentation. However, the proposed model still segments the cracks relatively completely and captures small cracks that other models often ignore (blue box).
Figure 6. Visualization of testing results on the DeepCrack. (a) original image, (b) groundtruth, (c) Swin-Unet, (d) TransUNet, (e) DeepCrack, (f) CrackNet. The red box denotes the areas that are erroneously identified as cracks. The blue box represents the fine cracks that are prone to being overlooked.
(2)
Qualitative results on the Crack500
To better demonstrate the advantages of the model proposed in this paper, we further selected experimental results from the Crack500 dataset, as shown in Figure 7. From the figure, it can be observed that the proposed model outperforms other models in terms of its resistance to interference from noisy textured backgrounds and its ability to accurately extract narrow cracks.
Figure 7. Visualization of testing results on the Crack500. (a) original image, (b) groundtruth, (c) Swin-Unet, (d) TransUNet, (e) DeepCrack, (f) CrackNet.
(3)
Qualitative results on the CFD
The experimental results on the CFD dataset are shown in Figure 8. From the figure, it is easy to observe that the proposed model demonstrates excellent detail extraction and noise resistance capabilities. For instance, in the first sample, the low contrast of the image significantly interferes with crack segmentation, but the proposed CrackNet successfully captures details that other methods overlook (highlighted in the red box). In the third sample, an error during image capture resulted in the photographer’s shoe being mistakenly included in the frame. Notably, the DeepCrack model, lacking long-distance modeling capabilities, incorrectly identifies the shoe as a crack (highlighted in the blue box).
Figure 8. Visualization of testing results on the CFD. (a) original image, (b) groundtruth, (c) Swin-Unet, (d) TransUNet, (e) DeepCrack, (f) CrackNet. The red box denotes the areas that are erroneously identified as cracks. The blue box represents the fine cracks that are prone to being overlooked.

4.7. Comparisons with Other Methods

The proposed method was also compared with other methods across two public datasets. The data for other methods were obtained from the best score of the original papers. The scale of the CFD dataset is very small and the crack regions in this dataset are extremely slender, posing a challenge to the model’s sensitivity in detecting fine-grained crack pixels. As shown in Table 4, our proposed CrackNet method achieved the best performance in terms of recall (Re) and F1 score, with a slightly lower precision rate (Pr) compared to other methods.
Table 4. Comparison of results between CrackNet and other methods on CFD.
In the DeepCrack dataset, crack regions only account for a small number of pixels, and the problem of category imbalance is particularly prominent. It can be seen from Table 5 that the overall performance of the proposed network is comparable with recent specialized crack segmentation algorithms and other segmentation algorithms, and is also superior to some other segmentation algorithms.
Table 5. Comparison of results between CrackNet and other methods on DeepCrack.

4.8. Ablation Experiments

In order to evaluate the validity of each module in the proposed model, an ablation study is performed on the CFD dataset. The experimental results are shown in Table 6. From the table, it can be seen that strip pooling has a positive effect on the model’s segmentation performance for small cracks. Due to the characteristics of small cracks, the contribution to the improvement in accuracy may not be significant, but it cannot be ignored. And the proposed dynamic loss function greatly improves the problem of class imbalance, which is reflected in a significant increase in recall rate.
Table 6. Results of the ablation experiments on the CFD.

5. Conclusions

The proposed CrackNet network combines the advantages of CNN and transformer, and introduces strip pooling to better capture the slender characteristics of cracks. More importantly, to address the issue of class imbalance, we have developed a dynamic loss function during the training process. Experiments on three public datasets demonstrate the superiority of the proposed method. In future work, we plan to further test our approach on actual crack images and consider deploying it on edge devices.
However, there are still some problems to be solved in the field, such as how to effectively use the edge information. Moreover, compared to other natural images, crack images exhibit high intra-class similarity and typically involve small dataset sizes, making them prone to overfitting with excessive convolutional layers. Therefore, using pruning techniques, such as reducing the number of convolutional layers, can not only simplify the structure but also potentially enhance performance. We will consider reducing the number of convolutional layers in our future research to make the model more feasible for practical and real-time industrial applications.

Author Contributions

Conceptualization, Y.F. and Y.S.; methodology, Z.H.; data curation, Q.L.; writing—original draft preparation, Y.F.; writing—review and editing, J.C. and Q.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research is partly fouded by the project ‘The Key Technologies and Applications of the Fujian Jiang Mainstream Flood Control Enhancement Project (Fuzhou Section) for Efficient, Safe, and Intelligent Management’ (824081116), National Natural Science Foundation of China (62476139) and Project of Sichuan Provincial Key Laboratory of Artificial Intelligence (2023RYY08).

Institutional Review Board Statement

Not applicable.

Data Availability Statement

We use three publicly available image datasets to evaluate the detection network proposed in this paper, namely, the CFD dataset, the Crack-2 dataset, and the Crack500 dataset. They can be downloaded from the URL https://paperswithcode.com/dataset/cfd (accessed on 30 June 2024), https://universe.roboflow.com/university-bswxt/crack-bphdr?ref=ultralytics (accessed on 5 February 2024) and https://paperswithcode.com/dataset/crack500 (accessed on 30 March 2024).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Mohan, A.; Poobal, S. Crack detection using image processing: A critical review and analysis. Alex. Eng. J. 2018, 57, 787–798. [Google Scholar] [CrossRef] [Scilit]
  2. König, J.; Jenkins, M.; Mannion, M.; Barrie, P.; Morison, G. What’s Cracking? A Review and Analysis of Deep Learning Methods for Structural Crack Segmentation, Detection and Quantification. arXiv 2022, arXiv:2202.03714. [Google Scholar]
  3. Pan, Y.; Zhang, X.; Cervone, G.; Yang, L. Detection of asphalt pavement potholes and cracks based on the unmanned aerial vehicle multispectral imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 3701–3712. [Google Scholar] [CrossRef] [Scilit]
  4. Dung, C.V.; Anh, L.D. Autonomous concrete crack detection using deep fully convolutional neural network. Autom. Constr. 2019, 99, 52–58. [Google Scholar] [CrossRef] [Scilit]
  5. Xinghao, C.; Wang, G.; Guo, H.; Zhang, C.; Wang, H.; Zhang, L. Mfa-net: Motion feature augmented network for dynamic hand gesture recognition from skeletal data. Sensors 2019, 19, 239. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  6. Wooram, C.; Cha, Y.-J. SDDNet: Real-time crack segmentation. IEEE Trans. Ind. Electron. 2019, 67, 8016–8025. [Google Scholar]
  7. Cui, X.; Wang, Q.; Dai, J.; Xue, Y.; Duan, Y. Intelligent crack detection based on attention mechanism in convolution neural network. Adv. Struct. Eng. 2021, 24, 1859–1868. [Google Scholar] [CrossRef] [Scilit]
  8. Girshick, R. Fast r-cnn. arXiv 2015, arXiv:1504.08083. [Google Scholar]
  9. Redmon, J. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
  10. Huyan, J.; Li, W.; Tighe, S.; Xu, Z.; Zhai, J. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection. Struct. Control Health Monit. 2020, 27, e2551. [Google Scholar] [CrossRef] [Scilit]
  11. Liu, Y.; Yao, J.; Lu, X.; Xie, R.; Li, L. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation. Neurocomputing 2019, 338, 139–153. [Google Scholar] [CrossRef] [Scilit]
  12. Yang, F.; Zhang, L.; Yu, S.; Prokhorov, D.; Mei, X.; Ling, H. Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans. Intell. Transp. Syst. 2019, 21, 1525–1535. [Google Scholar] [CrossRef] [Scilit]
  13. Schütt, K.T.; Sauceda, H.E.; Kindermans, P.J.; Tkatchenko, A.; Müller, K.R. Schnet–a deep learning architecture for molecules and materials. J. Chem. Phys. 2018, 148, 241722. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  14. Vijay, B.; Kendall, A.; Cipolla, R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar]
  15. Olaf, R.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention, Proceedings of the MICCAI 2015: 18th International Conference, Part III 18, Munich, Germany, 5–9 October 2015; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
  16. Wang, L.; Ye, Y. Computer vision-based road crack detection using an improved I-UNet convolutional networks. In Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China, 22–24 August 2020. [Google Scholar]
  17. Fangyu, L.; Wang, L. UNet-based model for crack detection integrating visual explanations. Constr. Build. Mater. 2022, 322, 126265. [Google Scholar]
  18. Fan, L.; Zhao, H.; Li, Y.; Li, S.; Zhou, R.; Chu, W. RAO-UNet: A residual attention and octave UNet for road crack detection via balance loss. IET Intell. Transp. Syst. 2022, 16, 332–343. [Google Scholar] [CrossRef] [Scilit]
  19. Lau, S.L.; Chong, E.K.; Yang, X.; Wang, X. Automated pavement crack segmentation using u-net-based convolutional neural network. IEEE Access 2020, 8, 114892–114899. [Google Scholar] [CrossRef] [Scilit]
  20. Chen, T.; Cai, Z.; Zhao, X.; Chen, C.; Liang, X.; Zou, T.; Wang, P. Pavement crack detection and recognition using the architecture of segNet. J. Ind. Inf. Integr. 2020, 18, 100144. [Google Scholar] [CrossRef] [Scilit]
  21. Shengyuan, L.; Zhao, X.; Zhou, G. Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Comput. Aided Civ. Infrastruct. Eng. 2019, 34, 616–634. [Google Scholar]
  22. Kang, D.; Benipal, S.S.; Gopal, D.L.; Cha, Y.J. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Autom. Constr. 2020, 118, 103291. [Google Scholar] [CrossRef] [Scilit]
  23. Honghu, C.; Wang, W.; Deng, L. Tiny-Crack-Net: A multiscale feature fusion network with attention mechanisms for segmentation of tiny cracks. Comput. Aided Civ. Infrastruct. Eng. 2022, 37, 1914–1931. [Google Scholar]
  24. Ashish, V.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems; Guyon, I., Von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Eds.; Neural Information Processing Systems Foundation, Inc. (NeurIPS): La Jolla, CA, USA, 2017. [Google Scholar]
  25. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
  26. Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 10–17 October 2021. [Google Scholar]
  27. Nikita, K.; Kaiser, Ł.; Levskaya, A. Reformer: The efficient transformer. arXiv 2020, arXiv:2001.04451. [Google Scholar]
  28. Wenjun, W.; Su, C. Automatic concrete crack segmentation model based on transformer. Autom. Constr. 2022, 139, 104275. [Google Scholar]
  29. Shamsabadi, E.A.; Xu, C.; Rao, A.S.; Nguyen, T.; Ngo, T.; Dias-da-Costa, D. Vision transformer-based autonomous crack detection on asphalt and concrete surfaces. Autom. Constr. 2022, 140, 104316. [Google Scholar] [CrossRef] [Scilit]
  30. Dang, L.M.; Wang, H.; Li, Y.; Nguyen, T.N.; Moon, H. DefectTR: End-to-end defect detection for sewage networks using a transformer. Constr. Build. Mater. 2022, 325, 126584. [Google Scholar] [CrossRef] [Scilit]
  31. Qi, H.; Kong, X.; Jin, Z.; Zhang, J.; Wang, Z. A Vision-Transformer-Based Convex Variational Network for Bridge Pavement Defect Segmentation. In Proceedings of the IEEE Transactions on Intelligent Transportation Systems, Edmonton, AB, Canada, 15–17 October 2024; Volume 25, pp. 13820–13832. [Google Scholar]
  32. Chen, J.; Lu, Y.; Yu, Q.; Luo, X.; Adeli, E.; Wang, Y.; Lu, L.; Yuille, A.L.; Zhou, Y. Transunet: Transformers make strong encoders for medical image segmentation. arXiv 2021, arXiv:2102.04306. [Google Scholar]
  33. Cao, H.; Wang, Y.; Chen, J.; Jiang, D.; Zhang, X.; Tian, Q.; Wang, M. Swin-unet: Unet-like pure transformer for medical image segmentation. In Computer Vision, Proceedings of the ECCV 2022 Workshops, Part III, Tel Aviv, Israel, 23–27 October 2022; Springer: Cham, Switzerland, 2023. [Google Scholar]
  34. Guo, X.; Lin, X.; Yang, X.; Yu, L.; Cheng, K.-T.; Yan, Z. UCTNet: Uncertainty-guided CNN-Transformer hybrid networks for medical image segmentation. Pattern Recognit. 2024, 152, 110491. [Google Scholar] [CrossRef] [Scilit]
  35. Wu, M.; Jia, M.; Wang, J. TMCrack-Net: A U-shaped network with a feature pyramid and transformer for mural crack segmentation. Appl. Sci. 2022, 12, 10940. [Google Scholar] [CrossRef] [Scilit]
  36. Xiang, C.; Guo, J.; Cao, R.; Deng, L. A crack-segmentation algorithm fusing transformers and convolutional neural networks for complex detection scenarios. Autom. Constr. 2023, 152, 104894. [Google Scholar] [CrossRef] [Scilit]
  37. Liu, H.; Miao, X.; Mertz, C.; Xu, C.; Kong, H. Crackformer: Transformer network for fine-grained crack detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 10–17 October 2021. [Google Scholar]
  38. Wang, J.; Zeng, Z.; Sharma, P.K.; Alfarraj, O.; Tolba, A.; Zhang, J.; Wang, L. Dual-path network combining CNN and transformer for pavement crack segmentation. Autom. Constr. 2024, 158, 105217. [Google Scholar] [CrossRef] [Scilit]
  39. Xie, S.; Tu, Z. Holistically-Nested Edge Detection. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 1395–1403. [Google Scholar]
  40. Jie, F.; Qu, B.; Yuan, Y. Distribution equalization learning mechanism for road crack detection. Neurocomputing 2021, 424, 193–204. [Google Scholar]
  41. Li, K.; Wang, B.; Tian, Y.; Qi, Z. Fast and Accurate Road Crack Detection Based on Adaptive Cost-Sensitive Loss Function. IEEE Trans. Cybern. 2021, 53, 1051–1062. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  42. Hou, Q.; Zhang, L.; Cheng, M.M.; Feng, J. Strip pooling: Rethinking spatial pooling for scene parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020. [Google Scholar]
  43. Shi, Y.; Cui, L.; Qi, Z.; Meng, F.; Chen, Z. Automatic road crack detection using random structured forests. IEEE Tran. Intel. Trans. Syst. 2016, 17, 3434–3445. [Google Scholar] [CrossRef] [Scilit]
  44. Mengyang, P.; Huang, Y.; Guan, Q.; Ling, H. Rindnet: Edge detection for discontinuity in reflectance, illumination, normal and depth. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 10–17 October 2021; pp. 6879–6888. [Google Scholar]
  45. Koonce, B. EfficientNet. In Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization; Apress: New York, NY, USA, 2021; pp. 109–123. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.