UCrack-DA: A Multi-Scale Unsupervised Domain Adaptation Method for Surface Crack Segmentation

Abstract

Surface cracks serve as early warning signals for potential geological hazards, and their precise segmentation is crucial for disaster risk assessment. Due to differences in acquisition conditions and the diversity of crack morphology, scale, and surface texture, there is a significant domain shift between different crack datasets, necessitating transfer training. However, in real work areas, the sparse distribution of cracks results in a limited number of samples, and the difficulty of crack annotation makes it highly inefficient to use a high proportion of annotated samples for transfer training to predict the remaining samples. Domain adaptation methods can achieve transfer training without relying on manual annotation, but traditional domain adaptation methods struggle to effectively address the characteristics of cracks. To address this issue, we propose an unsupervised domain adaptation method for crack segmentation. By employing a hierarchical adversarial mechanism and a prediction entropy minimization constraint, we extract domain-invariant features in a multi-scale feature space and sharpen decision boundaries. Additionally, by integrating a Mix-Transformer encoder, a multi-scale dilated attention module, and a mixed convolutional attention decoder, we effectively solve the challenges of cross-domain data distribution differences and complex scene crack segmentation. Experimental results show that UCrack-DA achieves superior performance compared to existing methods on both the Roboflow-Crack and UAV-Crack datasets, with significant improvements in metrics such as mIoU, mPA, and Accuracy. In UAV images captured in field scenarios, the model demonstrates excellent segmentation Accuracy for multi-scale and multi-morphology cracks, validating its practical application value in geological hazard monitoring.

1. Introduction

In geological disaster prevention and emergency response, ground surface cracks serve as early warning signs of potential hazards [1], revealing hidden safety risks in rock masses, soil bodies, and artificial structures. Accurate monitoring of crack distributions under complex surface structures is of paramount importance for safety prevention and geological disaster early warning. In recent years, the rapid advancement of unmanned aerial vehicle (UAV) technology has facilitated the widespread use of high-resolution UAV imagery in geological monitoring [2]. These images offer notable advantages, including wide spatial coverage, high temporal resolution, and efficient detection of crack distributions in complex surface environments.

Crack detection and segmentation, as fundamental computer vision tasks, have long been of interest in fields such as engineering safety and geological disaster prevention. Early methods primarily relied on traditional image processing techniques, such as Canny edge detection [3] and Hough transform [4], which were effective for regular cracks but tended to fail under complex backgrounds, varying lighting conditions, and noise interference. With the development of deep learning technologies, particularly the widespread application of CNNs [5], crack segmentation entered a data-driven end-to-end learning phase. Yang et al. [6] applied FCN [7] to crack detection tasks, achieving efficient pixel-level crack segmentation through end-to-end training. DeepCrack [8], based on VGG-16 [9], integrated batch normalization layers and side network prediction fusion techniques, significantly improving the Accuracy of FCN [7] in crack detection across various scenarios. CrackUNet [10], leveraging UNet [11]’s unique encoder–decoder symmetric skip connection structure, enhanced the detection of small-scale cracks by increasing the number of encoder layers, but this also demanded a larger number of training samples. DenseCrack [12], built on DenseNet [13], utilized the concept of dense connections to propose an improved deep architecture, demonstrating superior performance compared to other crack segmentation methods. With the introduction of the Transformer [14] architecture, it has found applications in crack detection. Liu H and colleagues proposed the Crackformer [15] based on the Transformer architecture, a model specifically optimized for fine-grained crack detection. Hatem Taha et al. [16] employed ConvNeXt [17] as the encoder and adopted the UPerNet [18] architecture as the decoder to learn both local and global semantic features of road cracks, significantly improving segmentation Accuracy. SCSegamba [19] utilizes the state space model (SSM) [20] to extract crack morphology and background texture information, enhancing the semantic continuity among crack pixels. SAM [21] is a general-purpose foundation model for image segmentation, with its core innovation lying in the prompt-based mechanism that enables zero-shot transfer capabilities, making it applicable to crack semantic segmentation tasks. Although large models demonstrate outstanding performance, their training typically requires massive amounts of data.

Despite the promising performance of existing crack segmentation algorithms on specific datasets, their generalization capability remains challenging when the sample size is small. Due to the characteristics of geological structures and limitations of imaging conditions, the distribution of crack regions in work areas is typically highly imbalanced [22]. In large-scale field scenes, cracks exhibit diverse morphologies and are locally densely distributed, which makes manual annotation challenging, leading to an extremely limited number of obtainable crack sample images. Supervised learning paradigms require a substantial amount of labeled data to establish semantic segmentation models, where a considerable number of labeled data are used to train models to handle a small amount of remaining data, significantly diminishing the marginal benefits of manual annotation.

In this context, unsupervised domain adaptation methods demonstrate unique application value. Currently, research on crack semantic segmentation largely relies on datasets collected from single surface types or ideal environments, such as pavements [16], concrete [23], and tunnels [24], which fail to fully reflect the complexity of various surface structures in real-world scenarios, including rocks, soil, and highways. As illustrated in Figure 1, domain shift exists among different crack datasets, manifested in the diversity of factors such as background noise, surface types, and crack quantity and morphology, as well as crack scales. Additionally, there are significant differences in shooting angles, lighting conditions, and surface details across different datasets during collection. These factors lead to a severe domain shift problem between crack semantic segmentation datasets, resulting in models trained on existing crack datasets performing poorly when directly applied to new datasets, failing to achieve the expected Accuracy.

In recent years, domain adaptation (DA) has emerged as an effective transfer learning technique and has been widely applied in fields such as object detection [26], medical image analysis [27], and remote sensing [28], aiming to alleviate the degradation of model performance caused by domain shift. Zhu et al. [29] used Generative Adversarial Networks (GANs) [30] to transform source domain images into synthetic images with the style of the target domain. Wang et al. [31] employed a category-adaptive threshold mechanism to generate pseudo-labels on the target domain for domain adaptation. When dealing with imbalanced sample categories, self-training methods can easily suffer from the “long-tail effect,” [32] causing the network’s performance on low-frequency categories to decline continuously. Hoffman et al. [33] were the first to propose the application of adversarial training to unsupervised domain adaptation for semantic segmentation. Srivastava et al. [34] achieved unsupervised domain adaptation for crack segmentation through incremental training with adversarial learning, without significantly reducing the Accuracy of the source domain. When applying adversarial training to unsupervised domain adaptation for semantic segmentation, the high-level features generated by the encoder are typically input into the discriminator, allowing the discriminator and encoder to engage in adversarial training to extract domain-invariant features across different datasets.

To extract domain-invariant features at multiple levels and enhance the Accuracy and robustness of crack semantic segmentation, the method designed in this paper combines a hierarchical adversarial mechanism and prediction entropy minimization constraints and utilizes an advanced semantic segmentation network to process multi-scale features. The contributions of this paper are summarized in the following three points:

• Design a hierarchical adversarial mechanism to extract domain-invariant features of different scales, combined with a prediction entropy minimization mechanism to sharpen decision boundaries, which improves the model’s domain adaptation capability for crack segmentation tasks in various scenes.
• To enhance the model’s ability to capture features of cracks at different scales and morphologies, we design a U-shaped crack semantic segmentation network. It extracts multi-scale receptive field features through stacking dilated convolutions with multiple dilation rates and optimizes the reconstruction of multi-morphological crack structures in the upsampling stage by combining mixed convolutional kernels.
• Construct a UAV ground surface crack dataset, containing a variety of complex factors in real-world scenes, which verifies the applicability of the proposed method on UAV images and provides important data support for the research on ground surface crack segmentation.

2. Related Work

2.1. Crack Semantic Segmentation

In recent years, crack segmentation techniques [6,8,10,12,15] based on deep learning have developed rapidly, significantly improving the Accuracy of crack recognition in specific scenarios. The overall workflow is shown in Figure 2. First, a pretrained crack semantic segmentation network is obtained by learning crack features from public datasets. Then, it is necessary to construct training samples for specific work areas, which involves manually annotating cracks in images to create a dataset for network transfer training. The training set is input into the initial network model to generate a preliminary segmentation result. Next, by comparing the model’s output with the true labels, the loss function is calculated, and the model parameters are optimized based on the loss values. This process continues to iterate until the model converges and achieves stable segmentation performance. Finally, the model with the best performance is selected for crack segmentation prediction.

Existing crack semantic segmentation methods rely heavily on supervised learning. When dealing with complex cracks accompanied by various noise interferences, they often require large-scale, high-quality annotated data for training support. However, within a single work area, regions containing cracks are limited, and the labeling process is cumbersome. A single crack image dataset may only contain a few hundred samples [35], making the data volume insufficient to support the transfer training of large models. In this context, the introduction of unsupervised domain adaptation methods is particularly crucial, significantly reducing the human resources needed for crack detection tasks.

2.2. Unsupervised Domain Adaptation

Domain adaptation aims to solve the problem of feature distribution differences between source and target domains, which has significant application value in semantic segmentation tasks. The current mainstream domain adaptation methods can be roughly divided into two categories: self-training-based methods [36,37] and adversarial learning-based methods [34,38,39], as illustrated in Figure 3.

Self-training methods generate pseudo-labels [40] by selecting high-confidence results from the target domain, which are then used as supervisory signals to fine-tune the model, thereby optimizing its performance on the target domain. The core advantage of such methods is their simplicity and efficiency, without the need for additional adversarial training modules. However, self-training methods face the challenge of the long-tail effect [32] in semantic segmentation tasks: the quality of pseudo-labels for small sample categories in the target domain is low, causing the model to fit high-frequency categories during training while neglecting the feature learning of tail categories. Adversarial learning methods reduce the impact of domain shift by introducing a domain discriminator, which encourages the model to learn domain-invariant features. Through adversarial training between the feature extractor and the discriminator, the feature distributions of the source and target domains are induced to converge. However, adversarial learning methods are prone to training instability issues, as the dynamic interaction between the generator and the discriminator can lead to gradient oscillation, a problem that is particularly pronounced when dealing with high-dimensional semantic segmentation tasks.

3. Method

3.1. Overview

To effectively extract domain-invariant features in both source and target domains and enhance the precision of crack semantic segmentation, we propose UCrack-DA, whose overall structure is shown in Figure 4. To strengthen the model’s ability to model the local details and overall shapes of cracks, we designed a U-shaped semantic segmentation network with a Mix-Transformer [41] backbone, specifically optimizing for the diversity in crack scales and morphologies. In the U-shaped network encoder, features produced at each downsampling step are concatenated with features of the same size during upsampling, meaning that the multi-scale features generated by the encoder all impact the model’s output. Traditional adversarial training methods usually only input the high-level features produced by the final stage of the encoder into the discriminator, generating domain-invariant features through adversarial training. To enable the encoder to produce multi-scale domain-invariant features, we designed a discriminator tailored to the Mix-Transformer [41] encoder, inputting multi-scale features into the discriminator for adversarial training to generate domain-invariant features at these scales. Unlike existing methods that only constrain the encoder to extract domain-invariant features, we train the decoder to learn target domain features and sharpen the decision boundary in the target domain by using the prediction entropy of the target domain images as a loss. Simultaneously, to enhance the stability of the domain adaptation method and reduce the network’s forgetfulness of crack features, we conduct supervised incremental learning on the source domain data.

The network training strategy proposed in this paper consists of two stages: the first stage involves the U-shaped semantic segmentation network fully learning from the source domain data; the second stage achieves unsupervised domain adaptation through adversarial learning between the encoder and the domain discriminator, along with entropy minimization constraints.

The semantic segmentation network employs an encoder–decoder architecture to achieve end-to-end crack segmentation. In the encoding phase, the Overlap Patch Embedding layer constructs an image embedding representation using convolution operations with overlapping regions. This is followed by hierarchical feature extraction through four cascaded Mix-Transformer [41] Blocks, progressively establishing feature representations from local details to global semantics, ultimately outputting four feature maps of different scales. The deepest feature map is processed by the Multi-Scale Dilation Attention Module (MSDAM) for multi-receptive field information enhancement, effectively fusing context information across different scales before being input into the decoder. The decoding phase uses a mixed convolutional decoder structure, which includes four Mixed Convolutional Attention Modules (MCAMs) and five Patch Expanding [42] operations for upsampling. Through skip connection mechanisms, the decoder deeply fuses features from corresponding levels of the encoder, effectively compensating for the multi-scale feature information that may be lost during downsampling. MCAM precisely reconstructs the geometric morphology of cracks by fusing features produced by heterogeneous convolution kernels through attention mechanisms, while Patch Expanding [42] gradually restores the resolution of the feature maps through channel expansion and feature rearrangement strategies, ultimately achieving high-precision pixel-level crack semantic segmentation.

3.2. Proposed Unsupervised Domain Adaptation Method

The overall workflow of the domain adaptation method is shown in Figure 5. The encoder and discriminator extract multi-scale domain-invariant features through hierarchical adversarial training. Meanwhile, to enable the decoder to learn target domain features and sharpen decision boundaries, a minimization prediction entropy constraint is imposed on the output results of the target domain. To enhance the stability of the training process, supervised training on the source domain data continues.

3.2.1. Hierarchical Adversarial Training

During adversarial training, we use multi-scale features as input for the discriminator, enabling it to effectively distinguish between the feature distributions of the source and target domains. At the same time, the encoder learns to generate feature representations that can confuse the discriminator through adversarial training. Specifically, this training mechanism adopts an alternating optimization strategy: first, the discriminator is trained with fixed encoder parameters to accurately determine the origin of the features; subsequently, the encoder is optimized with fixed discriminator parameters to generate features that can deceive the discriminator. Through this adversarial training paradigm, the encoder and discriminator continuously optimize in a dynamic game, ultimately achieving effective alignment of inter-domain features.

Source domain images $x_s\in {\mathcal{X}}_s$ and target domain images $x_t\in {\mathcal{X}}_t$ are input into the Mix-Transformer [41] encoder to extract features at four different scales. The source domain features are denoted as $F_s$, and the target domain features are denoted as $F_t$. The four sets of features are input into the discriminator, where they are first mapped to a shared embedding space [43] using four different MLP layers. Due to the different feature dimensions at each level, bilinear interpolation [44] is used to unify the resolution. After the feature dimensions are unified, channel concatenation and convolution operations are employed to fuse and compress the spatial dimensions. Finally, the multi-dimensional tensors are converted into one-dimensional tensors, and an MLP is used for domain classification, outputting the domain category corresponding to the features.

When training the discriminator, the label for source domain features is set to 0, and the label for target domain features is set to 1. The training objective function for the discriminator is as follows:

$$\underset{{\theta }_{DISC}}{min}{\displaystyle \frac{1}{\left|{\mathcal{X}}_s\right|}}\sum _{x_s}{\mathcal{L}}_D(F_s,0)+{\displaystyle \frac{1}{\left|{\mathcal{X}}_t\right|}}\sum _{x_t}{\mathcal{L}}_D(F_t,1)$$

(1)

where ${\theta }_{DISC}$ denotes the network parameters of the discriminator. We use BCE Loss [45] as the loss function ${\mathcal{L}}_D$ for domain classification, defined as follows:

$${\mathcal{L}}_D=-d_{i}log\left({\widehat{d}}_i\right)-(1-d_i)log(1-{\widehat{d}}_i)$$

(2)

where $d_i$ denotes the true domain label of the sample and ${\widehat{d}}_i$ denotes the output of the discriminator.

During the training of the encoder, to make the data distribution of the target domain features approximate that of the source domain features, only the target domain features are input into the discriminator, and the loss is calculated between the output domain label and the true domain label 0 of the source domain. Simultaneously, supervised learning is performed on the source domain data. The training objective function for the encoder is as follows:

$$\underset{{\theta }_{ENC}}{min}{\displaystyle \frac{{\lambda }_{adv}}{\left|{\mathcal{X}}_t\right|}}\sum _{x_t}{\mathcal{L}}_D(F_t,0)+{\displaystyle \frac{1}{\left|{\mathcal{X}}_s\right|}}\sum _{x_s}{\mathcal{L}}_{seg}(y_s,{\widehat{y}}_s)$$

(3)

where ${\theta }_{ENC}$ denotes the network parameters of the encoder, ${\lambda }_{adv}$ is the weight factor, ${\widehat{y}}_s$ is the pixel-level label of the source domain image, and $y_s$ is the network’s prediction. We use Weighted BCE Loss [46] and Dice Loss [47] as the loss functions for training on source domain data, defined as follows:

$$\begin{array}{c}{\mathcal{L}}_{Dice}=1-{\displaystyle \frac{2{\sum }_{i=1}^N{y}_i{\widehat{y}}_i+\epsilon }{{\sum }_{i=1}^N{y}_i+{\sum }_{i=1}^N{\widehat{y}}_i+\epsilon }}\hfill \\ {\mathcal{L}}_{WBCE}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left[w_1{y}_{i}log\left({\widehat{y}}_i\right)+w_2(1-y_i)log(1-{\widehat{y}}_i)\right]\hfill \\ {\mathcal{L}}_{seg}={\mathcal{L}}_{Dice}+{\mathcal{L}}_{WBCE}\hfill \end{array}$$

(4)

where N denotes the total number of pixels, $y_i$ represents the ground truth label values of the pixels, ${\widehat{y}}_i$ indicates the predicted values’ output by the network, $w_1$ and $w_2$ are the weights for background pixels and crack pixels, respectively, and $\epsilon =1\times {10}^{-30}$ is a numerical stability coefficient. During training, we alternately optimize objectives (1) and (3) to train the network to extract domain-invariant features from both the source and target domains.

3.2.2. Prediction Entropy Minimization

By optimizing the decoder parameters through the entropy loss on target domain data, the model is forced to produce high-confidence predictions for target domain samples, thereby sharpening decision boundaries, reducing class confusion, and improving the network’s performance on the target domain. Meanwhile, to enhance the robustness of training, supervised learning with source domain data is incorporated. The objective function is defined as follows:

$$\underset{{\theta }_{DEC}}{min}{\displaystyle \frac{{\lambda }_{ent}}{\left|{\mathcal{X}}_t\right|}}\sum _{x_t}{\mathcal{L}}_{ent}\left({\widehat{y}}_t\right)+{\displaystyle \frac{1}{\left|{\mathcal{X}}_s\right|}}\sum _{x_s}{\mathcal{L}}_{seg}(y_s,{\widehat{y}}_s)$$

(5)

where ${\theta }_{DEC}$ denotes the network parameters of the decoder, ${\lambda }_{ent}$ denotes the weighting factor, and ${\widehat{y}}_t$ refers to the output of the decoder for target domain features. Considering the pixel-level prediction nature of semantic segmentation tasks, a normalized entropy loss function ${\mathcal{L}}_{ent}$ is employed, which is defined as

$${\mathcal{L}}_{ent}={\displaystyle \frac{1}{HW{log}_{2}C}}\sum _{h,w=1}^{H,W}\sum _{c=1}^C{p}_{i,c}^{(h,w)}{log}_2(p_{i,c}^{(h,w)}+\epsilon )$$

(6)

where $p_{i,c}^{(h,w)}=softmax{\left(z_i^{(h,w)}\right)}_c$ denotes the predicted probability that the i-th sample at location $(h,w)$ belongs to class c, z represents the output value of the decoder, $\epsilon =1\times {10}^{-30}$ is a small constant used to prevent numerical overflow, C indicates the total number of classes, and ${log}_{2}C$ is the normalization factor that scales the loss value to the $\left[0,1\right]$ range and eliminates the influence of the number of classes on the loss.

3.3. Multi-Scale Dilated Attention Module

To address the challenges commonly encountered in complex crack semantic segmentation tasks, such as scale diversity and background noise, the final features extracted by the encoder are input into a Multi-Scale Dilated Attention Module (MSDAM). This module utilizes convolutions with different dilation rates to extract multi-scale features, while the attention mechanism is used to enhance crack features and suppress background noise. Its structure is shown in Figure 6.

The module constructs a progressive feature extraction path using a 4-level cascaded dilated convolution, with dilation rates increasing exponentially (d = 1, 2, 4, 8). The structural design of this module is inspired by Atrous Spatial Pyramid Pooling (ASPP) [48], whose improved variants [49,50,51] have been widely adopted for multi-scale feature extraction. Lei Hu et al. [52] applied the ASPP module to remote sensing image segmentation tasks with the same resolution as used in this study and demonstrated that an exponentially increasing dilation rate enhances the effectiveness of multi-scale feature extraction. Therefore, we adopted this design in our work. Under this structure, shallow convolutional kernels focus on capturing local texture features around crack edges, while deeper convolutional kernels leverage progressively dilated receptive fields to capture long-range continuity features of cracks. In the feature fusion stage, multi-scale feature maps are aggregated through channel concatenation, followed by the introduction of the CBAM [53] dual attention mechanism. The channel attention [54] weights automatically enhance the response strength of crack-related channels across scales, while the spatial attention [55] suppresses background noise and highlights the spatial distribution of crack pixels.

3.4. Mixed Convolutional Attention Module

In crack semantic segmentation tasks, the highly diverse shapes, scales, and orientations of cracks in natural scenes pose significant challenges for feature reconstruction. Cracks typically exhibit extreme geometric properties, with widths often spanning only a few pixels, while their lengths may extend across different regions of the image, forming complex spatial distribution patterns. To address this, we use the Mixed Convolutional Attention Module (MCAB) as the core component of the decoder. Its structure is shown in Figure 7. MCAB is a decoder module that integrates various types of convolutional kernels, with the core idea of extracting features using multi-scale, multi-morphology convolutional kernels and combining them with the CBAM [53] dual attention mechanism to handle the complex characteristics of cracks, such as diverse shapes, slender edges, and random orientations.

Structurally, the module consists of four parallel convolution branches, each using different combinations of convolution kernels: 1 × 1, 3 × 3, 3 × 7/7 × 3, and 5 × 11/11 × 5. Fixed-size convolution kernels struggle to simultaneously capture the diversity of cracks. Multi-morphology convolutional kernels effectively address the directionality and length diversity of cracks. Specifically, the 1 × 1 convolution is used for channel feature fusion, the 3 × 3 convolution extracts basic texture details and short-range crack orientations, while the 3 × 7 and 7 × 3 asymmetric kernels, with their differentiated receptive fields in the horizontal and vertical directions, specialize in capturing the orientation features of elongated cracks. The 5 × 11 and 11 × 5 large-scale kernels target more complex large cracks or branching structures, enhancing context integration by expanding the receptive field. After the outputs of the four branches are concatenated with the original input features, they are processed by the CBAM [53] attention module to enhance the features, dynamically increasing the response weights of crack-related channels and spatial positions while suppressing background noise.

4. Experimental Results and Analysis

4.1. Datasets

To evaluate the performance of the proposed method in cross-domain crack semantic segmentation tasks, we designed and created a new target domain dataset, UAV-Crack, and conducted experiments using an existing source domain dataset (CrackSeg9k) [35] and another target domain dataset (Roboflow-Crack) [56]. All data samples were uniformly resized to a resolution of 512 × 512. Figure 8 shows some representative sample images from the three datasets.

UAV-Crack is a custom-built dataset constructed in this study, collected from real-world field scenes captured by UAVs. It covers various realistic scenarios, including rock fractures, soil shrinkage cracks, and road alligator cracking. The dataset exhibits significant multi-scale object distribution, complex background interference, and diverse imaging conditions. It shows a noticeable domain shift from the source domain data, making it highly challenging and suitable for research and evaluation in cross-domain crack segmentation tasks.

The source-domain dataset, CrackSeg9k [35], integrates and refines crack data from multiple previous studies. The collected images cover a wide range of lighting conditions, viewing angles, and resolutions. The dataset includes various crack patterns—such as linear, branched, and reticular cracks—across more than ten types of substrate surfaces, including concrete and asphalt. Another target domain dataset, Roboflow-Crack [56], is sourced from the Roboflow platform and includes 886 crack images covering various surface materials. Its data distribution is relatively close to the source domain, making it suitable for testing the model’s adaptability under low domain shift conditions. The names of the datasets along with the number of training and validation/test samples are listed in

Table 1. Dataset split and sample statistics.

Dataset Name	Training Samples	Validation/Test Samples	Total
CrackSeg9K [35]	7332	1827	9159
Roboflow-Crack [56]	748	138	886
UAV-Crack	150	45	195

.

4.2. Implementation Details

The training process consists of two stages. The first stage aims to allow the semantic segmentation model to fully learn from the source domain data. During this process, to enhance the model’s generalization ability, data augmentation techniques such as random vertical flipping, random horizontal flipping, random rotation, and random color space transformation are applied to the training data. When learning from the source domain data, to address the class imbalance between cracks and background, the background pixel weight in the Weighted BCE Loss [46] is set to 1, while the crack pixel weight is set to 1.5. During training, the batch size is set to 8, the number of epochs is set to 100, and the initial learning rate ($l{r}_{\mathrm{init}}$) is set to $5\times {10}^{-5}$. The training process adopts a two-stage learning rate scheduling strategy: during the first five training epochs, the learning rate is gradually warmed up from the initial value to the initial learning rate in an exponential rise manner. In the subsequent training stages, a cosine annealing scheduling strategy [57] is applied, where the learning rate follows a cosine function with period T, decaying to a predefined minimum value ($l{r}_{\min }$), thus balancing the model’s convergence speed and Accuracy. The definition is as follows:

$$lr=\left\{\begin{array}{cc}l{r}_{min}\times {\left({\displaystyle \frac{l{r}_{\mathrm{init}}}{l{r}_{min}}}\right)}^{{\displaystyle \frac{t+1}{5}}}& 0\le t<5\\ l{r}_{min}+{\displaystyle \frac{l{r}_{\mathrm{init}}-l{r}_{min}}{2}}\left(1+cos\left(\pi \times {\displaystyle \frac{\left(t-5\right)\%T}{T}}\right)\right)& t\ge 5\end{array}\right.$$

(7)

In the second stage, while performing cross-domain feature alignment, incremental training on the source domain data is conducted to reduce model forgetting in the crack segmentation task and enhance the stability of the training process. During the adversarial training phase, the semantic segmentation network and the discriminator are optimized using different optimizers. The initial learning rates are set to $5\times {10}^{-5}$ and $2\times {10}^{-5}$, respectively, and cosine annealing is used to decay them to a lower bound of $1\times {10}^{-8}$. The weight factors ${\lambda }_{adv}$ and ${\lambda }_{ent}$ in the objective functions (3) and (5) are set to 0.001 and 0.0002, respectively.

All experiments in this study were conducted on an NVIDIA GeForce RTX 4090D, using PyTorch 2.1.0 to build, train, and test the model. Adam [58] was used as the optimization algorithm, with first-order and second-order momentum coefficients set to 0.9 and 0.999, respectively, and a weight decay coefficient of $1\times {10}^{-5}$.

4.3. Evaluation Metrics

Given the characteristics of multi-form, multi-scale crack semantic segmentation tasks in noisy environments, an evaluation system is constructed using three metrics: mean Intersection over Union (mIoU), mean Pixel Accuracy (mPA), and Accuracy. mIoU quantifies the spatial overlap between the predicted region and the true cracks, focusing on the model’s ability to recognize crack shape details. mPA evaluates pixel-level classification Accuracy from a class-balanced perspective, effectively revealing the model’s stability in distinguishing sparse crack pixels. Accuracy, as a basic metric, reflects the global pixel classification correctness. This multi-metric coupling strategy balances segmentation precision, class balance, and global consistency, making it particularly suitable for complex scenarios with strong lighting interference, shadow occlusion, and small crack distributions, providing a basis for evaluating the model’s reliability in real-world environments. Their computational expressions are as follows:

$$\begin{array}{c}mIoU={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \frac{|Y_i\cap {\widehat{Y}}_i|}{|Y_i\cup {\widehat{Y}}_i|}}\\ mPA={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \frac{|Y_i\cap {\widehat{Y}}_i|}{|Y_i|}}\\ Accuracy={\displaystyle \frac{{\displaystyle \sum _{i=1}^N}|Y_i\cap {\widehat{Y}}_i|}{{\displaystyle \sum _{i=1}^N}\left|Y_i\right|}}\end{array}$$

(8)

where N denotes the number of classes, $Y_i$ denotes the set of pixels that truly belong to class i, and ${\widehat{Y}}_i$ denotes the set of pixels predicted by the model as class i.

4.4. Ablation Study

We validate the effectiveness of the designed MSAdv (Multi-Scale Adversarial), MinEnt (Minimize Entropy), MSDAB, and MCAB through ablation experiments on UAV-Crack. The results of the ablation experiments are presented in

Table 2. Ablation study of each module combination.

	MSAdv	MinEnt	MSDAM	MCAM	Accuracy (%)	mPA (%)	mIoU (%)
Remove	✗	✓	✓	✓	96.55	68.28	63.27
	✓	✗	✓	✓	96.39	70.31	64.80
	✓	✓	✗	✓	96.33	70.97	64.96
	✓	✓	✓	✗	96.58	69.66	64.16
	✗	✓	✗	✓	96.20	68.48	62.33
	✗	✓	✓	✗	95.68	68.14	60.74
	✓	✗	✗	✓	96.51	67.45	63.51
	✓	✗	✓	✗	96.43	68.75	64.05
UCrack-DA (Ours)					96.31	71.76	65.33

The best results are shown in bold. ✓ indicates the module is included; ✗ indicates it is removed.

, and the quantitative metrics demonstrate that the various modules of UCrack-DA can effectively improve segmentation performance.

Removing MSAdv leads to a significant decrease in mIoU by 2.06% (65.33% vs. 63.27%), indicating that the multi-scale feature alignment strategy effectively enhances the feature consistency of the model in cross-domain scenarios. The primary role of MinEnt is to generate clearer decision boundaries in the decoder; its removal results in a 0.53% decrease in mIoU, suggesting it has a certain optimization effect on the segmentation results of the target domain data. The removal of the MSDAB and MCAB modules causes mIoU decreases of 0.37% and 1.17%, respectively, highlighting their importance in capturing multi-scale and multi-morphological cracks. Moreover, when multiple modules are removed simultaneously, the model performance further degrades compared to removing individual modules, which highlights the synergistic effect of the proposed unsupervised domain adaptation strategy and feature processing modules in enhancing segmentation performance. Notably, removing certain modules results in a slight drop in mIoU while yielding higher Accuracy, indicating that the model tends to make conservative predictions by assigning most pixels to the background class, thereby increasing overall Accuracy.

The complete model (UCrack-DA) achieves a high Accuracy of 96.31% while also attaining 71.76% mPA and 65.33% mIoU, indicating that each module collectively optimizes the model’s segmentation performance in domain adaptation scenarios through different mechanisms: MSAdv addresses inter-domain discrepancies, MinEnt sharpens the decision boundaries, further enhancing segmentation performance in the target domain, while MSDAB and MCAB enhance the model’s ability to represent multi-scale and multi-morphological cracks from a feature enhancement perspective and jointly suppress background noise through attention mechanisms.

4.5. Comparison with Other Methods

To validate the model’s ability to extract domain-invariant features across different levels of domain shift, we conducted experiments using CrackSeg9k [35] as the source domain data and Roboflow-Crack [56] and UAV-Crack as the target domain data. Among the compared domain adaptation methods, approaches based on different training paradigms can be categorized into two types: a) self-training methods, represented by DACS [36] and DAFormer [37], which achieve target domain knowledge transfer through iterative generation of pseudo-labels; b) adversarial training methods, such as AdaptSegnet [38], ADVENT [39], and CrackUDA [34], which utilize discriminative networks to align cross-domain feature distributions. All methods are evaluated under a unified experimental setting, including the use of identical data augmentation strategies, input resolution, batch size, optimizer, and learning rate scheduling scheme, with an MiT [41] encoder pretrained on ImageNet [59]. To comprehensively evaluate the domain adaptation performance of the proposed method, we additionally established two important benchmarks for comparison and visualized the segmentation results under these benchmarks: (1) Source Only: a standard model trained using only source domain data, representing the baseline performance of the model without domain adaptation. This result intuitively reflects the degree of distribution discrepancy between the source and target domains; (2) Oracle: an ideal model trained with full supervision on the target domain training set, representing the upper limit of the model’s performance on the target domain. This benchmark is used to measure the gap between the domain adaptation methods and the theoretical optimal solution.

4.5.1. Crackseg9K→Roboflow-Crack

Table 3. Experimental results on Roboflow-Crack.

Method	Accuracy (%)	mPA (%)	mIoU (%)
DACS [36]	96.08	70.77	66.93
DAFormer [37]	96.61	75.04	71.19
AdaptSegnet [38]	97.07	77.06	71.69
ADVENT [39]	97.27	79.66	73.84
CrackUDA [34]	97.60	82.17	76.95
UCrack-DA	97.92	90.90	81.34
Source Only	96.81	76.79	69.54
Oracle	98.71	93.26	87.19

The best results are shown in bold.

presents the metric scores of all methods on Roboflow-Crack [56]. Here, “Source Only” indicates the results of directly applying the model to the target domain without domain adaptation, while “Oracle” represents the results of training the model with supervised learning on the target domain. The results show that self-training models generally exhibit insufficient performance, primarily due to the “long-tail effect,” [32] which often leads to a continuous decline in performance for low-frequency categories in the target domain, especially when there is a severe imbalance between the number of crack and background categories in the samples. Our UCrack-DA outperforms all methods in comprehensive metrics, achieving an Accuracy of 97.92%, mPA of 90.90%, and mIoU of 81.34%. This represents a significant improvement compared to DAFormer [37], which uses the same feature extraction network and is based on a self-training approach. CrackUDA [34] also employs adversarial training to extract domain-invariant features, but it lacks task-specific optimization for crack segmentation and only extracts high-level domain-invariant features, resulting in limited performance.

In Figure 9, we present the segmentation results of the top three methods ranked by mIoU, selected from representative samples in the Roboflow-Crack [56] dataset. The visual comparison results are consistent with the quantitative metrics. By examining the first row, we can observe surface textures in the image that closely resemble cracks in appearance, making them highly misleading. ADVENT [39] misclassifies these textures as cracks due to its inability to suppress background noise. In contrast, our network is barely affected by such noise, as the proposed method leverages attention mechanisms to suppress background interference and enhance crack-related features. In the second row, the image features weak crack characteristics, which greatly tests the network’s feature extraction capability. By obtaining global features through the encoder and aggregating contextual information using MCAB and by minimizing prediction entropy in the target domain to sharpen decision boundaries, our network successfully identifies most of the cracks. The third and fourth rows represent large-scale and small-scale cracks, respectively, posing significant challenges to the network’s multi-scale feature processing ability. Other methods perform poorly on samples containing small-scale cracks due to their failure to extract multi-scale domain-invariant features. Our network combines multi-scale feature fusion and attention mechanisms to capture superior contextual information from the samples and enables the encoder to extract domain-invariant features at multiple scales during the domain adaptation stage, thereby comprehensively extracting small-scale cracks and optimizing the edges of large-scale cracks. In the fifth row, the image contains numerous cracks of varying sizes and shapes, which severely tests the network’s comprehensive capabilities. Through comparison, it is evident that our model maintains high Accuracy. Overall, compared to other domain adaptation schemes, UCrack-DA achieves better connectivity and Accuracy in segmenting cracks from images.

4.5.2. Crackseg9K→UAV-Crack

Table 4. Experimental results on UAV-Crack.

Method	Accuracy (%)	mPA (%)	mIoU (%)
DACS [36]	96.14	62.12	58.28
DAFormer [37]	95.58	68.11	60.49
AdaptSegnet [38]	95.82	66.77	61.13
ADVENT [39]	95.78	67.09	61.23
CrackUDA [34]	96.39	68.45	62.88
UCrack-DA	96.31	71.76	65.33
Source Only	96.21	63.18	59.15
Oracle	96.97	79.36	70.44

The best results are shown in bold.

presents the metric scores of all networks on the UAV-Crack dataset. This dataset is quite challenging, yet our approach still achieved the best results, with an Accuracy of 96.31%, mPA of 71.76%, and mIoU of 65.33%. CrackUDA has the highest Accuracy, but its core metric mIoU is very low. The Accuracy metric is more sensitive to the majority class (background), indicating that under this method, areas with weak crack features are often misjudged as background.

In Figure 10, we present the segmentation results of the top three methods ranked by the mIoU metric, selected from representative samples in the UAV-Crack test set. From the figure, it is evident that the difficulty of the samples in this dataset has significantly increased, as they contain a large amount of noise, various types of terrain, and cracks of multiple scales and morphologies. The network’s ability to resist interference and extract information about crack width is particularly crucial for improving performance metrics. The first two rows represent cases of mixed surfaces consisting of roads and soil. UCrack-DA achieves better segmentation results by enhancing crack-related semantic features and suppressing background noise through attention mechanisms. CrackUDA [34], with limited handling of crack features, results in poor continuity of the extracted cracks. ADVENT [39], lacking specific treatment for background noise, misclassifies some background textures as cracks. Rows three to five represent various soil surfaces with different degrees of vegetation coverage and relatively small-scale cracks. Other methods fail to detect some of the finer cracks due to the lack of multi-scale domain-invariant feature extraction. In contrast, UCrack-DA successfully identifies most cracks by extracting multi-scale domain-invariant features and enhancing the decision boundaries. Row six depicts a rocky surface with cracks of varying scales; our method not only detects the cracks but also preserves finer local details more effectively. The results demonstrate that, faced with these situations, UCrack-DA can extract multi-scale domain-invariant features from both the source and target domains to handle cracks of various forms and scales, and it suppresses noise through attention mechanisms.

4.5.3. Comprehensive Analysis

Through the validation of cross-dataset comparative experiments, the UCrack-DA method proposed in this study demonstrates significant advantages in tests on two target domain datasets. It is worth noting that although the mPA and mIoU metrics vary considerably across different models in our experiments, the overall Accuracy remains consistently high. In crack semantic segmentation tasks, cracks typically occupy only a small portion of the image, with background pixels far outnumbering crack pixels. As a result, overall pixel Accuracy tends to be dominated by the majority class (background), often leading to inflated Accuracy scores. In our dataset, crack pixels account for only 3.376% of the total pixels. Due to this extreme class imbalance, the performance on crack segmentation has a limited effect on the overall Accuracy, making it less sensitive to the model’s true capability in detecting cracks. Therefore, we primarily use mIoU as the core evaluation metric, while Accuracy and mPA serve as supplementary indicators to validate model performance. To demonstrate the domain adaptation capability of UCrack-DA, we randomly sample 180 images from each of the three datasets and extract features using the Mix-Transformer [41] encoder. These high-dimensional features are then reduced to two dimensions using t-SNE [60], resulting in the feature distribution shown in Figure 11a. Subsequently, CrackSeg9K [35] is designated as the source domain, while Roboflow-Crack [56] and UAV-Crack are used as target domains. Our method is applied to mitigate domain shift. The same samples are then encoded by the same encoder, and their features are reduced to two dimensions using the same t-SNE [60] method, producing the distribution shown in Figure 11b. It can be observed that the feature points from the three datasets are well mixed, indicating that the domain discrepancy has been significantly reduced. The MMD (Maximum Mean Discrepancy) [61] between the feature distributions of CrackSeg9K [35] and Roboflow-Crack [56] is reduced from 0.0335 to 0.0201, while the MMD [61] with UAV-Crack decreases from 0.0404 to 0.0241.

The experimental data indicates that the performance metrics of the method proposed in this paper surpass those of existing methods, and it maintains stable generalization capabilities across different data distributions. As visualized in Figure 9 and Figure 10, the method effectively suppresses complex background interference while accurately extracting multi-scale crack features and crack morphologies. The output results not only preserve the spatial coherence of the crack topology but also retain a significant amount of crack width information. This is primarily attributed to the hierarchical adversarial mechanism, which enables the encoder to generate multi-scale domain-invariant features. Meanwhile, the decoder sharpens decision boundaries and more effectively completes feature aggregation and upsampling.

4.6. Efficiency Analysis

To compare the model size and efficiency of different methods, we evaluate the number of parameters and computational complexity for each approach under a unified setting, using images with a resolution of 512 × 512 as input. For inference performance, one image is fed at a time, and each method is tested 10 times on the same hardware environment (NVIDIA RTX 3090 GPU), with the average inference time and standard deviation recorded. All results are summarized in

Table 5. Efficiency analysis of each method.

Method	Params (M)	FLOPs (GFLOPs)	Inference Time (ms)
DACS [36]	74.54	57.71	485.55 ± 9.68
DAFormer [37]	64.55	102.01	835.24 ± 10.8
AdaptSegnet [38]	74.54	57.71	489.88 ± 12.73
ADVENT [39]	74.54	57.71	486.35 ± 11.29
CrackUDA [34]	88.21	70.22	577.97 ± 9.35
UCrack-DA (ours)	97.64	73.36	652.13 ± 13.11

. Overall, our method achieves the best performance in crack semantic segmentation, with only slightly higher parameter count and computational cost.

5. Discussion

5.1. Limitations

The Transformer-based backbone network adopted in this study possesses strong modeling capabilities and effectively mitigates overfitting on the source domain. Its core mechanism is self-attention, with computational complexity. The network contains approximately 97.64 million parameters and incurs a forward inference cost of 73.36 GFLOPs. While it maintains high modeling capacity, it also comes with significant computational overhead, making it suitable for scenarios that demand high Accuracy and have ample computing resources. In remote sensing semantic segmentation tasks, the input is typically high-resolution imagery, which dramatically increases computational load and poses challenges for real-time performance. As a result, this model is not well-suited for deployment on edge devices such as drones.

5.2. Future Works

With targeted processing of multi-scale features and enhanced capability for detecting sparse objects, the innovative architecture of UCrack-DA demonstrates strong potential for extension to other remote sensing tasks, such as road and vehicle recognition. Future research can further investigate the application of unsupervised domain adaptation in remote sensing image analysis, particularly its adaptability across diverse scenarios like cross-region, cross-temporal, and cross-sensor conditions. Furthermore, incorporating more lightweight model architectures may reduce computational overhead, facilitating deployment on resource-limited edge devices and enhancing the model’s practicality, responsiveness, and stability in real-world settings.

6. Conclusions

This paper proposes a multi-scale unsupervised domain adaptation method for crack semantic segmentation, UCrack-DA, aimed at addressing the domain shift issue in surface crack segmentation in images and achieving high-quality segmentation results in the target domain. By designing a hierarchical adversarial mechanism and prediction entropy minimization constraints and integrating key modules such as the Mix-Transformer [41] encoder, MSDAM, and MCAM, UCrack-DA has achieved significant results in cross-domain feature alignment and crack segmentation in complex scenes. Experimental results demonstrate that the method performs exceptionally well on two target domain datasets, particularly in complex background and multi-scale crack segmentation, where its mIoU, mPA, and Accuracy significantly outperform existing methods.