High-Frequency Refined Mamba with Snake Perception Attention for More Accurate Crack Segmentation

Abstract

Cracks are vital warning signs to reflect the structural deterioration in concrete constructions and buildings. However, their diverse and complex morphologies make accurate segmentation challenging. Deep learning-based methods effectively alleviate the low accuracy of traditional methods, while they are limited by the receptive field and computational efficiency, resulting in suboptimal performance. To address this challenging problem, we propose a novel framework termed High-frequency Refined Mamba with Snake Perception Attention module (HFR-Mamba) for more accurate crack segmentation. HFR-Mamba effectively refines Mamba’s global dependency modeling by extracting frequency domain features and the attention mechanism. Specifically, HFR-Mamba consists of the High-frequency Refined Mamba encoder, the Snake Perception Attention (SPA) module, and the Multi-scale Feature Fusion decoder. The encoder uses Discrete Wavelet Transform (DWT) to extract high-frequency texture features and utilizes the Refined Visual State Space (RVSS) module to fuse spatial features and high-frequency components, which effectively refines the global modeling process of Mamba. The SPA module integrates snake convolutions with different directions to filter background noise from the encoder and highlight cracks for the decoder. For the decoder, it adopts a multi-scale feature fusion strategy and a strongly supervised approach to enhance decoding performance. Extensive experiments show HFR-Mamba achieves state-of-the-art performance in IoU, DSC, Recall, Accuracy, and Precision indicators with fewer parameters, validating its effectiveness in crack segmentation.

1. Introduction

Cracks serve as vital warning signs of structural deterioration in concrete constructions and buildings [1,2]. These fractures, often triggered by excessive loads, environmental erosion, and temperature variations, pose significant risks by undermining the structural integrity and compromising the safety of key infrastructures such as bridges and roads. Consequently, accurate crack segmentation plays an indispensable role in empowering timely repair and safeguarding public safety [3,4,5].

However, the long-distance distribution and varied morphologies of cracks pose a significant challenge. Specifically, cracks commonly exhibit elongated and irregular shapes in morphology. Different types of cracks share a common characteristic of strong contextual dependency, which requires the algorithms to consider the global connectivity of cracks. To address this challenging problem, traditional algorithms [6,7,8] primarily utilize classic image processing techniques to extract cracks, which are straightforward and efficient. Nevertheless, they often struggle with irregular morphology of cracks, which limits their effectiveness in complex real-world scenarios.

Recently, Artificial Intelligence has attracted widespread attention in various fields [9,10,11,12]. As one of the core techniques, deep learning is commonly used in the computer vision tasks including crack segmentation task. They can primarily be categorized into CNN-based networks and Transformer-based variants. CNN-based methods [13,14,15] can capture local crack-related features, while the limited receptive field poses a significant challenge when dealing with the diverse and complex crack patterns. Transformer-based methods [16,17,18] excel in modeling long-range dependencies, and thereby can capture global features of long-spanned cracks with high precision. Nevertheless, the quadratic computational complexity imposes a significant burden on Transformers, necessitating reliance on high-capacity hardware.

More recently, Mamba [19] has achieved significant success in the field of Natural Language Processing (NLP) due to its linear computational complexity and precise global modeling capabilities. Compared to CNN-based and Transformer-based models, Mamba-based methods [20,21,22] can achieve higher accuracy with lower computational complexity. However, these methods require converting the image into sequences and then modeling the global dependencies between the sequences by the state space model. As a result, during the conversion process, the contextual spatial information of the cracks is disrupted, leading to suboptimal performance in segmentation. Despite the loss of spatial information, the image-level semantics can still be explored from the frequency domain features. Motivated by this, we employ the Discrete Wavelet Transform (DWT) to explore the frequency domain features while recovering the lost spatial features, so as to refine the encoding process of Mamba.

In this paper, we aim to introduce the high-frequency features into Mamba to refine the spatial modeling process and enhance the performance of crack segmentation. Specifically, we use Discrete Wavelet Transform (DWT) to decouple the features into high-frequency and low-frequency components. Since the low-frequency components primarily contain background information, we discard these features. In contrast, we retain the high-frequency components, which contain a significant amount of texture features, helping to refine Mamba’s global representation of cracks. Notably, the high-frequency components obtained from DWT are in three different directions. To avoid unnecessary texture loss, we integrate these high-frequency semantics into new features. Subsequently, by injecting these high-frequency textures into Mamba’s encoding process, we effectively enhance the feature extraction accuracy of the encoder. In addition, considering that shallow features contain uncertain background noise, we specifically designed a Snake Perception Attention (SPA) module. By embedding this module into the skip connections between the encoder and decoder, it effectively filters out background-related noise. Meanwhile, by integrating snake convolutions in different directions with global max pooling operations, the SPA module highlights cracks effectively in both spatial and channel dimensions, thus enhancing the decoder’s performance.

In summary, this paper makes the following contributions:

• We propose a novel framework called HFR-Mamba, which effectively integrates frequency domain information to enhance Mamba’s performance in the spatial domain.
• The Refined Visual State Space (RVSS) module is embedded into the Mamba encoder to fuse high-frequency and spatial features for representative features. In addition, the decoder uses a strongly supervised approach and feature fusion to improve decoding performance.
• We introduce a novel Snake Perception Attention (SPA) module between the encoder and decoder, which can filter the background-related noise while highlighting the crack textures.
• Extensive experiments are conducted on three public datasets, and the results show that our proposed HFR-Mamba achieves the highest scores of IoU, DSC, Recall, Accuracy, and Precision metrics with fewer parameters.

2. Related Work

In this section, we review the development of crack segmentation and provide a brief overview of the Mamba-based networks in image segmentation.

2.1. Crack Segmentation Algorithms

Crack segmentation has been a fundamental problem and drawn sustained attention in civil engineering. Existing methods can be broadly categorized into two groups: traditional crack segmentation algorithms and deep learning-based crack segmentation networks. Traditional algorithms primarily rely on techniques such as threshold segmentation [6], detection operators [7], structure propagation [8], and morphological processing [23] to segment cracks. While these methods are fast and accurate, they are highly susceptible to interference from the image background, lighting, and noise, resulting in unstable segmentation performance. Consequently, our research focuses on deep learning-based crack segmentation methods.

Recently, deep learning techniques have been widely used in computer vision community [24,25]. Motivated by this, crack segmentation algorithms have extended the CNN framework to identify a wide variety of cracks. These CNN-based methods can be broadly classified into high-accuracy networks and real-time models. The former focuses on enhancing the contextual understanding of cracks by exploring effective feature aggregation strategies [13], designing robust attention modules [14], and developing feature extraction methods tailored to crack morphology [15]. For example, Yu et al. [15] designed a novel terrain surface crack detection network by analyzing the spatial characteristics of cracks and introducing adaptive deformable convolutions. On the other hand, some CNN-based methods focus on extending lightweight backbone networks with decoders and segmentation heads for real-time crack segmentation [26,27,28]. For instance, Zhang et al. [28] proposed an efficient crack segmentation network named ECSNet to accelerate real-time pavement crack segmentation by introducing small kernel convolutional layers and parallel max pooling. While these CNN-based models can segment cracks with high accuracy, their inherent limitations in the receptive field hinder their further development.

To address this issue, Transformers [16,17,18,29,30] have been introduced into crack segmentation. Wang et al. [16] proposed a novel SegCrack model by utilizing a hierarchical Transformer encoder and multi-scale feature fusion to segment cracks in diverse inspection scenarios. Shamsabadi et al. [18] proposed a Vision Transformer (ViT)-based framework for crack detection on asphalt and concrete surfaces by incorporating transfer learning and a differentiable Intersection over Union (IoU) loss function. A hybrid semantic segmentation model termed SCDeepLab [29] was proposed by integrating Swin Transformer [31] and CNN based on the DeepLabv3+ framework [32] for tunnel lining crack detection. These Transformer-based variants have demonstrated remarkable potential in crack segmentation tasks. However, the quadratic computational complexity of Transformers limits their application in general-purpose crack segmentation, particularly in real-time scenarios.

2.2. Mamba-Based Segmentation Methods

Recently, Mamba [19] has gained significant attention due to its achievements in the field of Natural Language Processing (NLP). Building on this success, its linear computational complexity and precise global dependency modeling have been investigated to enhance the efficiency and accuracy of image segmentation. Inspired by ViT [33], Zhu et al. [34] proposed Vision Mamba (Vim) for image segmentation. Vim employs patching and flattening operations to convert images into vectors, embeds positional information into these vectors, and uses bidirectional state space modeling (SSM) to capture long-range dependencies. Different from Vim, Liu et al. [35] introduced a novel VMamba by integrating a more efficient four-way scanning strategy into Visual State Space (VSS) blocks, which significantly improved the performance of Mamba in the field of computer vision [36]. However, the four-way scanning strategy is not universally effective, as different scanning orders may imply varying regional dependency relationships.

To address this issue, Fan et al. [37] proposed SliceMamba for medical image segmentation, ensuring that spatially adjacent features remain tightly coupled in the scanning sequence. For crack segmentation, Zuo et al. [21] also designed a task-specific scanning mechanism based on the morphological characteristics of cracks, achieving state-of-the-art performance. These Mamba-based variants achieve precise segmentation of specific targets by incorporating various scanning strategies. However, for diverse types of crack curves, focusing solely on the design of scanning strategies into Mamba proves insufficient to boost performance. Moreover, severe artifacts and noise interference intertwined with cracks pose significant challenges for achieving accurate crack segmentation.

3. Method

The proposed method adopts the classic encoder–decoder architecture to segment the cracks. It consists of a High-frequency Refined Mamba Encoder and Multi-scale Feature Fusion Decoder. To mitigate the impact of background noise, we specially design a novel Snake Perception Attention Module between the encoder and decoder to filter noise and highlight crack-related textures. The architecture of the proposed method is shown in Figure 1.

3.1. High-Frequency Refined Mamba Encoder

Mamba is efficient in modeling long-range dependency with linear computational overhead. However, the sequential scanning mechanism makes it easy for Mamba to encounter discontinuities when extracting elongated cracks, leading to the formation of suboptimal representations. Therefore, we fully explore the semantic information in the image from the frequency domain perspective, proposing the High-frequency Refined Mamba Encoder to effectively model long-range dependencies in cracks.

Specifically, the proposed encoder mainly consists of a high-frequency decoupling branch and a spatial modeling branch. Given the input image $X\in {\mathbb{R}}^{H\times W\times 3}$, it is sent in parallel to the spatial modeling branch and the high-frequency decoupling branch. For the high-frequency decoupling branch, it utilizes the Discrete Wavelet Transform (DWT), which can reduce the image resolution while preserving the textures. Note that the first high-frequency decoupling branch adopts two cascaded DWT operations, while the others use only one. Based on the DWT operation and $3\times 3$ convolution, we can disentangle the high-frequency component and obtain $X_d^f\in {\mathbb{R}}^{\frac{H}{4}\times \frac{W}{4}\times C_1}$ to further refine Mamba, which can be reformulated as:

$$\begin{array}{c}\hfill X_d^f=Conv\left(Merge\left(DWT\left(X\right)\right)\right),\end{array}$$

(1)

where $Merge(·)$ is the operation of fusing high-frequency components. For the spatial modeling branch, motivated by [35], it primarily uses a downsampling operation to reduce the image resolution and Refined Visual State Space (RVSS) module for global modeling. In the encoder, the first downsampling module performs downsampling at a 4× rate, while the others perform at a 2× downsampling rate. After the first downsampling module, we obtain $X_d^s\in {\mathbb{R}}^{\frac{H}{4}\times \frac{W}{4}\times 3}$, which is then sent to the RVSS module for global modeling, which can be calculated as:

$$\begin{array}{c}\hfill X_d^s=RVSS(Down\left(X\right), X_d^f),\end{array}$$

(2)

where $RVSS(·)$ represents the RVSS module. Different from the Visual State Space module [35], the RVSS module can model global dependencies while simultaneously introducing high-frequency features to enhance texture representation. The RVSS module is comprised of Layer Normalization, a High-frequency Fused SS2D (HF-SS2D) module, and a Feed-Forward Network (FFN). Layer Normalization is employed to standardize the data distribution, and the FFN is used to integrate features. The RVSS module adopts a sequential scanning strategy combined with a state space module to model global dependencies, while incorporating high-frequency textures to further refine the global modeling. Similar to the Visual State Space module [35], the RVSS module adopts a four-directional scanning strategy to capture sequential relationships across different patches, as illustrated in Figure 2:

By repeatedly applying the above operations, we can refine the encoding efficiency of Mamba and output hierarchical multi-scale features $\{X_{d_i}^s, i \in [1, 4]\}$ (where i represents the encoding stage).

3.2. Snake Perception Attention Module

Fusing multi-scale features of the encoder and decoder through skip connections is a commonly used technique in the encoder–decoder architecture [38]. It can complement deep features with spatial textures and improve segmentation performance. However, for crack images, due to numerous uncertain interference factors in the shooting environment, directly merging shallow and deep features may cause the continuity of cracks to be disrupted by noise. Therefore, inspired by the morphological structure of cracks, we specifically design a Snake Perception Attention (SPA) module to filter noise and highlight texture information from both spatial and channel dimensions, as shown in Figure 3.

The SPA module consists of three branches, including the channel excitation branch, the snake convolution branch, and the skip connection branch. The channel excitation branch primarily utilizes global max pooling and the ReLU activation function to integrate important channel feature information into the snake convolution branch, which can be reformulated as:

$$\begin{array}{c}\hfill X_i^{\prime }=ReLU\left(GMP\left(X_{d_i}^s\right)\right), i\in [1, 3]\end{array}$$

(3)

For snake convolution [39], due to its strong feature extraction capability for curved structures, we apply it to the spatial feature extraction of cracks. Specifically, given the layer feature $X_{d_i}^s$ from the encoder, we use two snake convolutions in different directions to extract features along the horizontal and vertical axes, respectively. We assume that the center of the snake convolution kernel K is $(X_h, X_v)$ and its receptive field size is $3\times 3$. The generation process of the receptive field of snake convolution in the horizontal and vertical directions can be calculated as:

$$K_{h\pm c}=\left\{\begin{array}{cc}\hfill (x_{h+c}, y_{h+c})& =(x_h+c, y_h+\sum _h^{h+c}\Delta y),\hfill \\ \hfill (x_{h-c}, y_{h-c})& =(x_h-c, y_h+\sum _h^{h-c}\Delta y),\hfill \end{array}\right.$$

(4)

$$K_{v\pm c}=\left\{\begin{array}{cc}\hfill (x_{v+c}, y_{v+c})& =(x_v+c, y_v+\sum _v^{v+c}\Delta y),\hfill \\ \hfill (x_{v-c}, y_{v-c})& =(x_v-c, y_v+\sum _v^{v-c}\Delta y),\hfill \end{array}\right.$$

(5)

where $\Delta$ represents the offset and $\Delta \in [-1, 1]$. c is the receptive field range and ∑ means the accumulation process. By combining two snake convolutions in different directions, we can effectively detect crack branches in different orientations. The skip connection branch adds the input features to the enhanced features, helping to alleviate the vanishing gradient problem. The whole process of the SPA module can be reformulated as:

$$\begin{array}{c}\hfill X_{SPA}^i=X_{d_i}^s+(Snak{e}_h\left(X_{d_i}^s\right)+Snak{e}_v\left(X_{d_i}^s\right))\odot X_i^{\prime },\end{array}$$

(6)

where ‘⊙’ is element-wise multiplication.

3.3. Multi-Scale Feature Fusion Decoder

To enhance the decoding performance, we design a Multi-scale Feature Fusion Decoder to restore feature resolution and decode crack semantics. The proposed decoder consists of three decoding stages. Each stage employs two cascaded $3\times 3$ convolution operations, a ReLU activation function, and Batch Normalization to integrate the upsampled deep features, which can be reformulated as:

$$\begin{array}{c}\hfill X_{u_3}=BN\left(ReLU\left(Conv\left(Up(X_{d_4}^s+SPA\left(X_{d_3}^s\right))\right)\right)\right),\end{array}$$

(7)

where $SPA(·)$ is the Snake Perception Attention module. Based on repeating the above process, the decoder can generate multi-scale features $\{X_{u_t}, t\in [1,4]\}$ (when $t=4$, $X_{u_4}$ is the result output by the proposed model). Before feature integration, the decoder employs bilinear interpolation to restore the resolution. The factors of the first three upsampling operations are set to $2\times$, while the last upsampling operation uses a factor of $4\times$, corresponding to the downsampling operations in the encoder. After feature integration, each stage of the decoder computes the loss by comparing the layer outputs with the downsampled ground truth, which can be reformulated as:

$$\begin{array}{c}\hfill Los{s}_{ce}=Cross\_Entropy(X_{u_t}, Down\left(GT\right)), t\in [1, 3]\end{array}$$

(8)

where $GT$ means the ground truth. Based on this strongly supervised approach, the decoder can effectively improve the final segmentation accuracy.

3.4. Loss Function

In this paper, we primarily use the Dice loss to supervise the coverage between the segmentation results and the ground truth. Motivated by [40], we employ Cross-Entropy loss to encourage the accuracy during the segmentation process to closely approximate the ground truth. Therefore, the total loss function can be reformulated as follows:

$$\begin{array}{cc}\hfill Los{s}_{total}& =Los{s}_{dice}(Pred, GT)+\lambda ·\sum _{t=1}^{3}Los{s}_{ce}(Pre{d}^t, G{T}^t),\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill Los{s}_{dice}& =1-{\displaystyle \frac{2·{\sum }_{i=1}^{N}Pre{d}_i·G{T}_i}{{\sum }_{i=1}^{N}Pre{d}_i^2+{\sum }_{i=1}^{N}G{T}_i^2}},\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill Los{s}_{ce}& =-{\displaystyle \frac{1}{N}}\sum _{i=1}^{N}G{T}_i·log\left(Pre{d}_i\right)+(1-G{T}_i)·log(1-Pre{d}_i),\hfill \end{array}$$

(11)

where i represent the i-th pixel and N means the total pixels of the image. The details of the Dice loss and Cross-Entropy loss are shown in Equations (10) and (11), respectively.

4. Experiments

4.1. Datasets

Rissbilder Dataset [41]. The Rissbilder dataset contains 2736 images of different types of cracks collected from buildings such as walls and bridges, along with their corresponding label images. The original resolution of the images in the dataset is $512\times 512$. The proportion of cracks in the images is approximately 2.70%, and some of the images have distortion issues. In the experiment, we randomly divided the dataset into training, validation, and test sets in a 6:1:3 ratio.

CrackTree200 Dataset [42]. The CrackTree200 dataset contains 175 images of cracks collected from pavement and building surfaces, along with manually annotated label images. The original resolution of the images is $800\times 600$. The proportion of crack pixels in the images is relatively low, approximately 0.31%. Additionally, the images in this dataset contain interference factors such as shadows, occlusions, low contrast, and noise. For the division of the training, validation, and test sets, we followed a 6:1:3 ratio.

GAPS384 Dataset [43]. The GAPS384 dataset contains 383 crack images collected during the summer of 2015 under dry and warm environmental conditions. Due to the specific constraints applied to the image collection environment, the dataset has minimal image noise caused by harsh weather conditions such as rain glare and mud obstruction. However, the dataset is still challenging due to random thickening distortions of the cracks. The original resolution of the images is $640\times 540$. Since the proportion of crack pixels in the images is 1.21%, it is classified as a low-crack-ratio dataset.

4.2. Performance Metrics

To comprehensively evaluate the proposed method, referenced by [44,45], we selected five commonly used metrics in semantic segmentation to quantify the differences between the prediction and the ground truth. Specifically, we chose $IoU$ and $DSC$ indicators to calculate the overlap ratio between the prediction and the ground truth, which can be calculated as:

$$\begin{array}{cc}\hfill IoU& ={\displaystyle \frac{TP}{TP+FN+FP}},\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill DSC& ={\displaystyle \frac{2·TP}{(TP+FN)+(TP+FP)}},\hfill \end{array}$$

(13)

where $TP$, $FP$, $TN$, $FN$ are true positives, false positives, true negatives, and false negatives, respectively. We also compute the mean IoU

($mIoU$) for parts of the datasets, which is obtained by averaging the IoU of the foreground and background. To focus on the pixels that are missed, we also introduce the $Recall$, Precision $\left(Pre\right)$, and Accuracy $\left(Acc\right)$ metrics, which can be reformulated as:

$$\begin{array}{cc}\hfill Recall& ={\displaystyle \frac{TP}{TP+FN}},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill Precision& ={\displaystyle \frac{TP}{TP+FP}},\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill Accuracy& ={\displaystyle \frac{TP+TN}{TP+FN+FP+TN}},\hfill \end{array}$$

(16)

4.3. Implementation Details

All experiments were conducted on an Ubuntu 22.04 system. The hardware supporting this system includes an Intel i5-12600KF CPU, an NVIDIA 3090 GPU, and a 1200 W power supply. Additionally, the software supporting the training process includes Anaconda version 22.3.1, PyTorch framework version 2.0, and PyCharm version 2023.1 (community version). Before the training, we applied data augmentation strategies to improve the model’s performance, including random flipping, random cropping, and random deformation. For the hyperparameters in the experiments, we set the batch size to 4, resized all images to a resolution of $400\times 400$, initialized the learning rate to 0.0001, set the total number of training iterations to 80,000, and selected the Adam optimizer to promote model convergence. We set the regularization parameter $\lambda$ in the loss function to 0.4. We set the weights for the fusion factor $\psi$ of the high-frequency branch and spatial branch in the encoder to 0.6, respectively.

4.4. Comparison Experiments

To evaluate the segmentation performance of our proposed HFR-Mamba, we compared it with eight state-of-the-art methods as competitors including U-Net [46], DeepLabv3 [47], Vision Transformer [33], Swin Transformer [31], CrackSegNet [48], DECSNet [49], RHACrackNet [50], and CarNet [51] across the public Rissbilder, CrackTree200, and GAPS384 datasets. Among these competitors, U-Net, DeepLabv3, Vision Transformer, and Swin Transformer are commonly used benchmark methods for general semantic segmentation, while the CrackSegNet, DECSNet, RHACrackNet, and CarNet models are state-of-the-art methods specifically designed for crack segmentation tasks. To explore the performance of these methods, we recorded their results with and without loading of the pretrained weight from the ImageNet-1K dataset [52]. Note that our proposed HFR-Mamba was not pretrained.

The performances of HFR-Mamba and other competitors on the Rissbilder dataset are recorded in

Table 1. Performance (%) of the methods on the Rissbilder dataset [41].

Method	Pretrained	mIoU	DSC	Recall	Acc	Pre
U-Net [46]	−	30.87	45.16	49.22	93.30	40.04
DeepLabv3 [47]	−	28.76	43.24	47.54	93.07	39.66
ViT [33]	−	56.51	68.62	71.29	98.53	63.51
Swin [31]	−	60.32	71.29	74.58	98.82	67.87
CrackSegNet [48]	−	55.49	68.78	70.37	98.29	62.74
DECSNet [49]	−	51.57	65.45	67.10	97.71	58.60
RHACrackNet [50]	−	60.12	70.57	70.95	98.46	67.34
CarNet [51]	−	60.86	71.92	71.74	98.98	68.29
HFR-Mamba	−	65.52	75.29	78.57	99.40	71.37
U-Net [46]	✓	32.33	47.57	51.18	93.59	43.52
DeepLabv3 [47]	✓	30.72	45.26	49.81	93.83	41.30
ViT [33]	✓	58.22	69.94	73.07	98.74	64.99
Swin [31]	✓	62.53	72.85	76.11	99.09	69.13
CrackSegNet [48]	✓	57.78	70.02	71.87	98.61	64.81
DECSNet [49]	✓	53.36	66.89	68.92	98.25	59.94
RHACrackNet [50]	✓	62.39	72.54	76.01	98.95	69.06
CarNet [51]	✓	62.81	73.04	76.29	99.21	69.33
HFR-Mamba	✓	66.87	76.53	79.64	99.52	72.58

. We observed that our proposed HFR-Mamba achieves the SOTA performance on all metrics. In terms of CarNet, which ranked second and was specifically designed for crack segmentation, our proposed HFR-Mamba model outperformed it by 2.71%, 2.25%, 2.28%, 0.19%, and 2.04% in the mIoU, DSC, Recall, Acc, and Precision metrics, respectively. Additionally, we found that loading pretrained weights from the ImageNet-1K dataset significantly improves the model’s segmentation performance, suggesting that pretraining on large-scale datasets can enhance the model’s accuracy in segmenting crack structures. To provide a more intuitive comparison, we visualized the segmentation results of HFR-Mamba and the top-performing models, as shown in Figure 4. From the comparison, we observed that HFR-Mamba is able to capture the fine-grained structures of cracks more accurately. In contrast, other methods, such as the Swin Transformer model, demonstrate suboptimal performance in segmenting curved areas, suggesting that general segmentation models require specialized techniques when dealing with cracks. Overall, our proposed HFR-Mamba model can accurately segmenting crack structures on the Rissbilder dataset.

To evaluate the performance of the proposed method on the CrackTree200 dataset, we recorded the scores of all models on the IoU, DSC, Recall, Accuracy, and Precision metrics. The CrackTree200 dataset primarily consists of tree-like crack images, which pose a challenge for models requiring global dependency modeling. As shown in

Table 2. Performance (%) of the methods on the CrackTree200 dataset [42].

Method	Pretrained	IoU	DSC	Recall	Acc	Pre
U-Net [46]	−	61.53	70.10	72.46	98.74	65.45
DeepLabv3 [47]	−	60.41	68.73	71.12	98.50	63.39
ViT [33]	−	72.26	76.51	80.60	99.19	74.33
Swin [31]	−	74.61	78.88	82.53	99.37	76.08
CrackSegNet [48]	−	71.67	75.47	80.33	99.08	74.14
DECSNet [49]	−	67.30	72.36	76.82	98.81	70.63
RHACrackNet [50]	−	74.20	78.53	80.09	99.35	75.97
CarNet [51]	−	75.13	79.85	83.87	99.50	76.22
HFR-Mamba	−	79.34	81.02	87.30	99.83	78.94
U-Net [46]	✓	62.84	71.90	74.27	98.96	66.29
DeepLabv3 [47]	✓	62.33	71.56	73.93	98.83	66.02
ViT [33]	✓	74.30	78.29	82.18	99.29	75.25
Swin [31]	✓	76.52	79.71	84.27	99.56	76.88
CrackSegNet [48]	✓	72.79	76.86	81.05	99.12	74.58
DECSNet [49]	✓	68.91	74.61	78.22	98.95	72.34
RHACrackNet [50]	✓	76.21	78.86	82.67	99.54	76.63
CarNet [51]	✓	76.69	79.85	84.39	99.62	76.84
HFR-Mamba	✓	80.59	82.46	89.41	99.89	80.07

, our HFR-Mamba achieved an IoU score of 79.34%, a DSC score of 81.02%, a Recall score of 87.30%, an Accuracy score of 99.83%, and a Precision score of 78.94%, outperforming all other models and achieving state-of-the-art performance across all metrics. On a broader scale, the segmentation performance of Transformers outperforms that of CNNs, validating our hypothesis that extracting crack contextual features requires long-range dependency modeling. To provide a deeper comparison of the segmentation performance, we visualized the results of the proposed HFR-Mamba and the top-performing segmentation models. As shown in Figure 5, the proposed HFR-Mamba accurately captures the tree-like crack structures, demonstrating its ability to model the relationships between crack structures and indicating its strong contextual modeling capability. In contrast, CarNet, which is specifically designed for crack segmentation, focuses solely on spatial feature extraction and neglects the advantages of frequency domain representations in capturing textures. As a result, it performs suboptimally in segmenting fine-grained features.

The segmentation performance of all models on the GAPS384 dataset is analyzed both qualitatively and quantitatively in

Table 3. Performance (%) of the methods on the GAPS384 dataset [43].

Method	Pretrained	mIoU	DSC	Recall	Acc	Pre
U-Net [46]	−	40.35	55.28	57.67	98.47	52.18
DeepLabv3 [47]	−	39.63	53.30	55.96	98.23	50.39
ViT [33]	−	64.36	74.73	76.17	98.89	71.34
Swin [31]	−	67.40	76.28	78.49	99.12	73.63
CrackSegNet [48]	−	62.47	71.97	74.68	98.71	70.51
DECSNet [49]	−	60.20	67.33	70.87	98.53	68.88
RHACrackNet [50]	−	67.16	75.89	78.11	99.07	72.02
CarNet [51]	−	68.12	78.23	79.70	99.29	75.15
HFR-Mamba	−	71.27	81.88	84.31	99.52	77.58
U-Net [46]	✓	42.44	57.06	60.16	98.69	53.94
DeepLabv3 [47]	✓	41.85	55.72	57.60	98.32	50.51
ViT [33]	✓	66.62	75.98	77.90	98.95	72.27
Swin [31]	✓	68.11	77.47	79.91	99.20	74.14
CrackSegNet [48]	✓	63.85	72.76	76.29	98.92	71.58
DECSNet [49]	✓	61.80	68.08	71.84	98.83	69.90
RHACrackNet [50]	✓	68.03	77.21	79.69	99.11	73.88
CarNet [51]	✓	68.98	79.26	80.80	99.34	75.82
HFR-Mamba	✓	72.61	83.25	85.94	99.61	79.22

and Figure 6. By comparing with other methods, we found that the proposed HFR-Mamba achieved the highest scores across all metrics. Specifically, HFR-Mamba obtained 71.27%, 81.88%, 84.31%, 99.52%, and 77.58% in mIoU, DSC, Recall, Accuracy, and Precision, respectively. Compared to the dedicated segmentation model CarNet, HFR-Mamba outperforms it by 2.29%, 2.62%, 3.51%, 0.18%, and 1.76% in mIoU, DSC, Recall, Accuracy, and Precision, respectively. Furthermore, for the general segmentation method Swin Transformer, HFR-Mamba surpasses it by 3.16%, 4.41%, 4.40%, 0.32%, and 3.44% in mIoU, DSC, Recall, Accuracy, and Precision, respectively. By comparing these models designed for different purposes, the superiority of the proposed method is effectively validated. Figure 6 visualizes the segmentation results of the proposed HFR-Mamba and the top-performing models. By comparison, we observed that HFR-Mamba exhibits a clear advantage in crack segmentation. It is capable of recognizing slender curves while preserving key fine-grained features. Therefore, the proposed method shows great potential for application in crack detection across other scenarios.

To provide a comprehensive comparison of all methods, we calculated their model parameters, FLOPs, FPS, and other efficiency-related metrics. As shown in

Table 4. The parameter counts, FLOPs value, and FPS of the proposed HFR-Mamba and competitors.

Method	Year	Param (M)	FLOPs (G)	FPS
U-Net [46]	2015	29.8	16.5	157.4
DeepLabv3 [47]	2017	23.6	36.6	143.4
ViT [33]	2020	50.2	101.7	207.5
Swin [31]	2021	37.6	82.2	242.0
CrackSegNet [48]	2020	18.0	157.6	194.1
DECSNet [49]	2024	47.4	149.5	72.3
RHACrackNet [50]	2024	4.6	26.3	329.2
CarNet [51]	2024	5.1	23.1	340.8
HFR-Mamba	−	22.1	21.9	280.9

, the proposed HFR-Mamba has 22.1 M parameter counts. Compared to Transformer-based models, such as ViT, the Mamba-based approach shows a clear advantage. Although the number of parameters in HFR-Mamba is not the lowest, our aim in this work is to achieve more precise segmentation. Additionally, the proposed HFR-Mamba method has a FLOPs value of 21.9 G, indicating its higher computational efficiency. Regarding the FPS metric, the proposed method is only slightly lower than the lightweight CarNet, suggesting that HFR-Mamba demonstrates good real-time performance. Thus, our proposed HFR-Mamba can achieve more accurate segmentation performance with lower computational overhead.

4.5. Ablation Study

In this subsection, we conducted ablation experiments on the individual components of the proposed HFR-Mamba to evaluate their effectiveness. Given the complexity of the crack structures in the CrackTree200 dataset, all ablation experiments are performed on this dataset. First, we removed the high-frequency refinement branch from the encoder and used the remaining part as the backbone. As shown in

Table 5. Ablation study for components of the proposed method on the CrackTree200 dataset. ‘w/’ and ‘w/o’ mean with and without, respectively.

Method	IoU (%)
Backbone	76.74
Backbone + high-frequency feature	77.82
Backbone + high-frequency feature + decoder	78.36
Backbone + high-frequency feature + SPA + decoder (w/o supervision)	79.02
Backbone + high-frequency feature + SPA + decoder (w/ supervision)	79.34

, by directly upsampling the segmentation results from the backbone, we achieved an IoU score of 76.74%. After incorporating the high-frequency features into the encoder, we observed a 1.08% increase in the IoU score. This validates the necessity of introducing high-frequency information into Mamba for crack segmentation. Next, we added the decoder. By comparing with the direct upsampling method, we found that the progressive decoder significantly improves crack segmentation, achieving a 0.54% increase in IoU. Additionally, we verified the contribution of the proposed SPA module. By comparing the performance of HFR-Mamba before and after adding the module, we found that the SPA module increased the IoU score by 0.66%. To visually assess the high-frequency branch, we performed a heatmap visualization of HFR-Mamba before and after adding the branch. As shown in Figure 7, we observed that the high-frequency branch enables HFR-Mamba to capture changes in the fine-grained branches of cracks and improves the global continuity. Thus, these results demonstrate that each component of HFR-Mamba plays a crucial role in crack segmentation tasks.

We also conducted the corresponding ablation study for the hyperparameter $\lambda$. As shown in Table 5, adding extra supervision to the decoder’s layer features led to a further improvement. We also conducted ablation experiments on the weights of the supervisory branch in the decoder. As shown in Figure 8, through comparison, we found that when $\lambda$ is set to 0.4, HFR-Mamba performs best across the three public datasets. This is because the higher weight for Dice Loss encourages the model to focus more on the overall integrity and consistency of the target region, which performs better for imbalanced data. Meanwhile, the lower weight for Cross-Entropy Loss ensures the proposed model maintains accuracy in boundary details without overfitting, thus improving the overall region segmentation. This setting helps enhance the overall performance of crack segmentation tasks, especially when dealing with complex backgrounds.

To explore the optimal fusion strategy of spatial and frequency domain features, we compared three methods: pixel-wise addition, tensor multiplication, and weighted fusion. As shown in

Table 6. Ablation study for the fusion manner of high-frequency and spatial features on the CrackTree200 dataset. Note that ‘★’ represents the proposed method.

Method	IoU	DSC	Recall	Acc	Pre
Multiplication	78.82	80.07	86.45	99.71	78.13
Addition	78.95	80.29	86.62	99.78	78.29
Weighted fusion (EFS)	79.03	80.32	86.66	99.80	78.35
Weighted fusion (WFES)	79.08	80.49	86.82	99.78	78.79
Proposed (★)	79.34	81.02	87.30	99.83	78.94

, we observed that by multiplying the aforementioned features, HFR-Mamba achieved IoU, DSC, Recall, Accuracy, and Precision scores of 78.82%, 80.07%, 86.45%, 99.71%, and 78.13%, respectively. Next, we experimented with adding the two feature sets together and found a slight improvement in segmentation accuracy. Subsequently, we employed a weighted fusion strategy to combine the two feature sets. We compared three strategies: emphasizing the frequency domain while weakening the spatial features (EFWS), equalizing the frequency and spatial features (EFS), and weakening the frequency domain while emphasizing the spatial features (WFES). We set the weight of the high-frequency branch as $\psi$. As shown in Figure 9, when $\psi \le 0.5$, the proposed method shows relatively lower performance on the three public datasets. This indicates that the fusion approach, which amplifies spatial features while reducing the importance of frequency domain features, is not suitable for crack segmentation tasks. This is because the spatial domain may contain background and noise, and amplifying these features can interfere with the semantics in the high-frequency features. In contrast, when $\psi >0.5$, the proposed method performs better on the three public datasets. This once again confirms the necessity of introducing high-frequency features. Additionally, we observed that when $\psi =0.6$, the performance of the proposed method was better than when $\psi =0.8$. This suggests that excessively amplifying frequency domain features while compressing spatial features can also negatively affect the segmentation performance. Since we only introduce high-frequency features while discarding low-frequency features, this inevitably leads to the loss of some background-related semantics, thereby impacting the model’s ability to recognize the contextual features of the cracks. Therefore, when $\psi =0.6$, the encoder is able to effectively fuse spatial features and high-frequency semantics, improving the segmentation performance.

To evaluate the performance of the SPA module, we conducted ablation experiments on it. Specifically, we compared its structure with three commonly used attention modules: the Squeeze-and-Excitation (SE) attention module [53], the Strip Pooling module [54], and the self-attention module [55] on the CrackTree200 dataset. As shown in

Table 7. Ablation study for the SPA module on the CrackTree200. ‘★’ means the proposed method.

Method	IoU (%)	DSC (%)	Recall (%)	Acc (%)	Pre (%)
SE	78.21	80.34	86.67	99.69	78.24
Strip Pooling	78.94	80.60	86.93	99.73	78.67
Self-attention	79.13	80.94	87.08	99.79	78.81
SPA (★)	79.34	81.02	87.30	99.83	78.94

, replacing the SPA module with the SE module resulted in a decrease of 1.13%, 0.68%, 0.63%, 0.14%, and 0.70% in IoU, DSC, Recall, Accuracy, and Precision, respectively. This demonstrates that simply increasing the model’s attention to channel features is not the optimal approach for crack segmentation. Next, we compared two spatial attention methods, and found that the Strip Pooling and self-attention modules significantly improved the model’s segmentation performance over the channel attention module. Finally, we compared these methods with the proposed SPA module. By observation, we found that our method achieved state-of-the-art performance across all metrics. This indicates that incorporating both channel and spatial attention in the HFR-Mamba model is well-suited for crack segmentation tasks.

Additionally, we conducted ablation experiments on the resolution of the input images. As shown in

Table 8. Ablation study for the resolution of input images on the CrackTree200 dataset. Note that ‘★’ represents the proposed method.

Resolution	IoU (%)	DSC (%)	Recall (%)	Acc (%)	Pre (%)
$128\times 128$	77.21	79.54	85.67	99.69	77.24
$224\times 224$	77.84	79.60	86.37	99.70	77.97
$256\times 256$	78.73	80.34	86.68	99.72	78.31
$400\times 400$ (★)	79.34	81.02	87.30	99.83	78.94

, we resized the original images to $128\times 128$, $224\times 224$, $256\times 256$, and $400\times 400$. From the comparison, we found that resizing the images to small resolutions significantly impacted the model’s accuracy in crack segmentation. This suggests that excessive downsampling disrupts the contextual relationships between crack structures, making precise segmentation more challenging. Due to memory constraints, we did not experiment with resolutions higher than $400\times 400$. Therefore, among all the tested resolutions, we found that $400\times 400$ can effectively maximize the potential of HFR-Mamba for crack segmentation.

5. Conclusions

This study focuses on addressing the challenge of accurate crack segmentation in concrete constructions and buildings. To address this challenging problem, we propose a High-frequency Refined Mamba with Snake Perception Attention Module (HFR-Mamba), which can refine the global modeling of Mamba and alleviate the problems of limited receptive field and insufficient computational overhead in deep learning methods. We have verified that mining high-frequency features from the frequency domain is one of the effective methods to improve the efficiency of Mamba encoding. To explore the fusion of frequency domain features and the high-dimensional features of Mamba encoding, we investigated their integration strategies. The results demonstrated that the optimal approach is to emphasize the frequency domain while weakening the spatial features, which is a superior feature fusion method. Additionally, the proposed Snake Perception Attention (SPA) module fully utilizes the dynamic snake convolution to capture features of curvilinear structures. Corresponding experiments have shown that, compared to commonly used general attention modules, this module achieves superior performance in crack segmentation tasks.

Limitations: Although the proposed algorithm achieves SOTA performance, the lack of hardware has prevented us from deploying the algorithm to drone systems or embedded systems to evaluate its performance in real-world scenarios. Additionally, there is still room for improvement in the model’s real-time performance.