DEPANet: A Differentiable Edge-Guided Pyramid Aggregation Network for Strip Steel Surface Defect Segmentation

Abstract

The steel strip is an important and ideal material for the automotive and aerospace industries due to its superior machinability, cost efficiency, and flexibility. However, surface defects such as inclusions, spots, and scratches can significantly impact product performance and durability. Accurately identifying these defects remains challenging due to the complex texture structures and subtle variations in the material. In order to tackle this challenge, we propose a Differentiable Edge-guided Pyramid Aggregation Network (DEPANet) to utilize edge information for improving segmentation performance. DEPANet adopts an end-to-end encoder-decoder framework, where the encoder consisting of three key components: a backbone network, a Differentiable Edge Feature Pyramid network (DEFP), and Edge-aware Feature Aggregation Modules (EFAMs). The backbone network is designed to extract overall features from the strip steel surface, while the proposed DEFP utilizes learnable Laplacian operators to extract multiscale edge information of defects across scales. In addition, the proposed EFAMs aggregate the overall features generating from the backbone and the edge information obtained from DEFP using the Convolutional Block Attention Module (CBAM), which combines channel attention and spatial attention mechanisms, to enhance feature expression. Finally, through the decoder, implemented as a Feature Pyramid Network (FPN), the multiscale edge-enhanced features are progressively upsampled and fused to reconstruct high-resolution segmentation maps, enabling precise defect localization and robust handling of defects across various sizes and shapes. DEPANet demonstrates superior segmentation accuracy, edge preservation, and feature representation on the SD-saliency-900 dataset, outperforming other state-of-the-art methods and delivering more precise and reliable defect segmentation.

1. Introduction

Strip steel is a vital component of the steel industry, widely used in automobile manufacturing, home appliance production, and construction structures [1]. During manufacturing, strip steel may develop various surface defects, such as scratches, cracks, oxide scales, pits, roll marks, and scabs [2]. These defects not only degrade the appearance but also potentially compromise the mechanical properties and service life of the material [3]. For instance, scratches and cracks can weaken the material’s strength, while oxide scales and scabs may negatively impact subsequent processing steps, leading to quality degradation [4]. Therefore, precise and prompt segmentation and the effective addressing of these defects are essential for guaranteeing product quality [5].

Advancements in deep learning have led to remarkable progress in segmenting steel surface defects [6]. For example, Liu et al. [7] proposed the multiscale context segmentation network to segment multiscale defects on strip steel surfaces. Built on Faster R-CNN, the network incorporates a parallel atrous convolution architecture to effectively capture multiscale context and minimize confusion. Du et al. [8] proposed AFF-Net for strip steel defect segmentation, tackling challenges from size variations, complexity, and diversity. AFF-Net introduces AFF-Block for adaptive feature weighting of segments and Foc-FPN to improve multiscale fusion and semantic balance.

As mentioned above, deep learning approaches have made remarkable strides in strip steel surface defect segmentation. However, according to the available information, these approaches have failed to fully recognize a crucial role of edge information in strip steel surface defect segmentation. In fact, in other practical applications, edge information has proven to greatly boost the model’s segmentation accuracy. For example, Dong et al. [9] presented CGTD-Net, a dual-branch network utilizing a channel-wise global transformer for strip steel defect segmentation. They enhanced edge information with the SCGA module and optimized feature fusion using the CGT module, improving the accuracy of small or narrow-edge defect segmentation. Hui et al. [10] proposed a model called EMC-UNet to enhance the accuracy of pulmonary nodule segmentation. EMC-UNet utilizes the Sobel operator to enhance edge information, improving the perception of pulmonary nodule boundaries, and employs a multiscale feature extraction module to capture features at different scales.

Based on the discussion above, we propose a Differentiable Edge-guided Feature Pyramid Aggregation Network (DEPANet) within an encoder–decoder framework, leveraging edge information from strip steel surface defects to enhance segmentation performance. This edge information inherently exhibits distinctive characteristics, such as intensity discontinuities and boundary contrasts, which can be exploited to guide precise segmentation. The encoder comprises three key components: a backbone network, a Differentiable Edge Feature Pyramid Network (DEFP), and Edge-aware Feature Aggregation Modules (EFAMs). Considering that ResNet possesses both strong feature extraction capabilities and high execution efficiency, we selected ResNet34 [11] as the backbone to extract the overall features of the input image. In contrast, DEFP is proposed to improve the model’s ability to extract edge components, consisting of a series of stacked Adaptive Feature Smoothing Filters (AFSFs). Each AFSF leverages a learnable Laplacian operator for edge feature extraction and a learnable Gaussian blurring mechanism for feature smoothing and enhancement. The multiple proposed EFAMs are employed to generate edge-enhanced features, with each EFAM designed to integrate the backbone-extracted global features along with the edge components from DEFP at the corresponding level. A Convolutional Block Attention Module (CBAM) is employed to ensure the effective integration of both global information and fine-grained edge details, thereby enhancing the network’s ability to accurately segment defects. For the decoding stage, a Feature Pyramid Network (FPN) is employed to extract multi-level features and fuse them hierarchically, ensuring precise defect localization and robustly handling defects of different sizes and shapes.

The main innovations of this paper are as follows:

• We propose a novel DEFP network consisting of stacked AFSFs to extract multiscale edge components from strip steel surface defects. Each AFSF utilizes a learnable Laplacian operator for edge feature extraction and a learnable Gaussian blurring mechanism for feature smoothing and enhancement.
• We propose EFAMs to generate edge-enhanced features, with each EFAM designed to integrate the backbone features with the edge components obtained from DEFP using a CBAM to enhance defect segmentation accuracy.
• Based on DEFP and EFAM, a novel network called DEPANet is proposed. Experimental results show that DEPANet achieves superior performance in segmenting and localizing complex defects on strip steel surfaces.

The organization of this paper is as follows. Section 2 reviews previous studies. Section 3 elaborates on the proposed methodology, encompassing the overall architecture, DEFP, EFAMs, and the loss function (Section 3.4). Section 4 provides the experimental results, and Section 5 concludes this study.

2. Related Work

2.1. ResNet-Based Methods

ResNet-based methods have been widely adopted for strip steel surface defect segmentation, particularly due to their ability to learn very deep architectures while overcoming the problem of vanishing gradients [12]. For example, Fan et al. [13] introduced ResAt-UNet, an enhanced U-Net to build image segmentation, which leverages ResNet’s residual module and attention mechanisms to boost segmentation accuracy. Sahli et al. [14] developed a ResNet-based glioma segmentation method and support vector machines to segment and classify tumor regions, improving diagnostic efficiency and accuracy. Abdelrahman et al. [15] developed a kidney tumor diagnosis model that combines a squeeze-and-excitation ResNet with a pyramid network feature for improved performance. Yang et al. [16] introduced FRPNet, a lightweight network for real-time semantic segmentation in unstructured scenes. FRPNet employs an enhanced Faster-ResNet encoder and a Partial Spatial Pyramid Pooling (PASPP) module to accurately extract semantic information and address irregular boundaries and similar categories.

2.2. Edge Enhancement

Edge enhancement plays a critical role in image segmentation, especially for tasks such as defect detection [17] and medical image segmentation [18]. Traditional edge enhancement techniques often rely on methods like Sobel, Canny, or Laplacian operators to highlight the edges, but they struggle in complex environments with low signal-to-noise ratios. In recent years, deep learning-based techniques have gained widespread recognition, excelling over traditional methods by automatically learning hierarchical edge features from data. Models such as U-Net and its variants have demonstrated great promise in enhancing edge details through multiscale feature extraction and refinement. For example, Allah et al. [19] introduced Edge U-Net, a deep convolutional neural network that improves the precision of brain tumor segmentation through integration of edge information. Jin et al. [20] proposed ESGNet, a fusion network combining multiple modalities for semantic segmentation of high-resolution remote sensing images. ESGNet improves segmentation accuracy by digital surface models with an edge segmentation guidance module and a multi-modal adaptive fusion module. Wang et al. [21] proposed a new approach for muscle shape modeling and inference, which improves segmentation accuracy and efficiency by combining probabilistic shape modeling with deep edge segmentation. In contrast, our proposed DEPANet achieves edge-aware segmentation through two novel modules: (1) DEFP for enhancing spatial detail via multi-scale edge feature propagation, and (2) EFAM for adaptive fusion of semantic and edge information without external edge annotations. Unlike CGTD-Net and EMC-UNet, DEPANet focuses on efficiently integrating edge and semantic features under purely internal supervision, achieving comparable or superior accuracy on challenging datasets with lower computational cost.

Existing methods have advanced strip steel defect segmentation as mentioned above, but they often overlook the critical role of edge information in improving accuracy. DEPANet addresses this issue by introducing a novel Differentiable Edge Feature Pyramid (DEFP) network and Edge-aware Feature Aggregation Modules (EFAMs) within an encoder–decoder framework. Unlike traditional approaches, DEFP dynamically extracts and refines multiscale edge features using learnable Laplacian operators and Gaussian blurring mechanisms. EFAMs then integrate these edge-cues with global backbone features, ensuring precise defect localization and robust performance across varying defect sizes and shapes.

3. Methods

3.1. Overall Architecture

Inspired by the encoder–decoder framework, we propose a novel Differentiable Edge-guided Feature Pyramid Aggregation Network (DEPANet) to aggregate multi-scale features from different stages, as shown in Figure 1.

The encoder consists of three key components: a backbone network, a Differentiable Edge Feature Pyramid Network (DEFP), and Edge-aware Feature Aggregation Modules (EFAMs). We selected ResNet34 [11] as the backbone to extract global characteristics from the steel surface due to its strong feature representation and efficient computational efficiency. Let $x\in R^{W\times H\times 3}$ be the input image, ResNet34 generates five level outputs, i.e., $f_i^d$, where $i=1,2,3,4,5$ represents the i-th layer. The DEFP network consists of five AFSFs, denoted ${\mathrm{AFSF}}_i$ for $i=1,2,3,4,5$. Each AFSF applies a learnable filtering operation to generate features $f_i^{bran}$ and $f_i^l$. Formally,

$$f_i^l,f_i^{bran}=\mathrm{AFSF}\left(f_{i-1}^l\right)$$

(1)

while $f_0^l=x$. The EFAMs integrate the overall features $f_i^d$ obtained from the backbone network with the edge information $f_i^l$ from the DEFP to obtain edge-enhanced feature $f_i^e$, where $i=1,2,3,4,5$. Formally,

$$f_i^e=\mathrm{EFAM}\left(f_i^d,f_i^l\right)$$

(2)

In the decoder, we employ the Feature Pyramid Network (FPN) [22] to progressively upsample and fuse edge enhanced multiscale features $f_i^e$, ultimately restoring high-resolution segmentation results for precise localization of steel surface defects of various sizes and shapes. Formally,

$$f_i^{\prime }=\mathrm{Fuse}\left(\mathrm{Upsample}\left(f_{i+1}^e\right),f_i^e\right),i=1,2,3,4$$

(3)

where $\mathrm{Upsample}(·)$ denotes the upsampling operation, and $\mathrm{Fuse}(·)$ represents the feature fusion operation.

Our method was proposed within the framework of an end-to-end encoder–decoder architecture for steel surface defect segmentation. The major contributions of this paper include the overall architecture, named the DEFP network, and the proposed EFAM modules. More details about DEFP and EFAM are presented in Section 3.2 and Section 3.3, respectively.

3.2. Differentiable Edge Feature Pyramid Network (DEFP)

The DEFP network is proposed to extract multiscale edge features through a series of Adaptive Feature Smoothing Filters (AFSFs), as shown in Figure 1. Each AFSF employs a learnable Laplacian operator for edge feature extraction and a learnable Gaussian blur mechanism for feature smoothing and enhancement, as shown in Figure 2.

The input feature $f_{i-1}^l$ is first processed through a learnable Gaussian blur module [23] to smooth the feature and suppress high-frequency noise, resulting in the blurred feature $f_i^{blur}$. Formally,

$$f_i^{blur}=\mathrm{GaussianBlur}\left(f_{i-1}^l+{\theta }_i\right)$$

(4)

where ${\theta }_i$ represents the parameter of the Gaussian blur. $f_i^{blur}$ is downsampled to obtain low-frequency information, denoted $f_i^{bran}$. Formally,

$$f_i^{bran}=\mathrm{Downsample}\left(f_i^{blur}\right)$$

(5)

$f_i^{bran}$ is the input of the next AFSF (${\mathrm{AFSF}}_{i+1}$). In addition, $f_i^{bran}$ undergoes an upsampling operation, followed by a novel learnable Gaussian blur module to further refine the feature map. Formally,

$$f_i^{refine}=\mathrm{GaussianBlur}\left(\mathrm{Upsample}\left(f_i^{bran}\right)\right)$$

(6)

Finally, the difference between the original input feature $f_{i-1}^l$ and the processed feature $f_i^{refine}$ is calculated to obtain the edge information $f_i^l$.

$$f_i^l=f_{i-1}^l-f_i^{refine}$$

(7)

$f_i^l$ is used for feature fusion in the EFAM.

3.3. Edge-Aware Feature Aggregation Modules (EFAMs)

The edge information and fine details are helpful for defect segmentation tasks [24]. Therefore, we propose EFAMs to integrate the edge components extracted by DEFP into the global features obtained from the backbone network, ensuring a comprehensive representation of both structural and detailed information. Figure 3 shows the architecture of EFAM.

Each EFAM, ${\mathrm{EFAM}}_i$, takes the backbone network’s output $f_i^d$ and DEFP’s edge feature $f_i^l$ at the i-th level as the inputs, and fuses them through an addition operation to obtain $f_i^{add}$:

$$f_i^{add}=f_i^l+f_i^d$$

(8)

The $f_i^{add}$ is then concatenated with the $f_i^d$, followed by a convolutional layer. Formally,

$$f_i^{cat}=\mathrm{Conv}\left(\mathrm{Concat}\left(f_i^{add},f_i^d\right)\right)$$

(9)

Then, a spatial attention mechanism is employed to weight the important regions of the aggeragated feature $f_i^{cat}$. In particular, a convolutional operation is performed on $f_i^{cat}$, followed by a Sigmoid, to obtain the weight matrix a:

$$a=\sigma \left(\mathrm{Conv}\left(f_i^{cat}\right)\right)$$

(10)

where $\sigma (·)$ is the Sigmoid function. a is used to weight the $f_i^{cat}$ to obtain the feature with significant representation. Formally,

$$f_i^{att}=a\otimes f_i^{cat}$$

(11)

where ⊗ indicates the element-wise multiplication. The feature $f_i^{att}$ is aggregated with the overall feature $f_i^d$ obtained from the backbone, obtaining a enhanced feature $f_i^{en}$, and a Convolutional Block Attention Module (CBAM) is applied to $f_i^{en}$ to further enhance feature representation by focusing on both channel-wise and spatial-wise attention. Formally,

$$f_i^e=\mathrm{CBAM}\left(f_i^d+f_i^{att}\right)$$

(12)

The details of CBAM are shown in Figure 3(bottom). CBAM integrates channel attention and spatial attention in a cascaded manner. CBAM first applies channel attention to the input $f_i^{en}$. Specifically, $f_i^{en}$ undergoes both max pooling as well as average pooling, followed by multilayer perceptrons (MLPs) and an element-wise multiply operation to generate the channel attention feature $f_i^{ca}$.

$$f_i^a=\mathrm{MLP}\left(\mathrm{MaxPool}\left(f_i^{en}\right)\otimes \mathrm{MLP}\left(\mathrm{AvgPool}\left(f_i^{en}\right)\right)\right)$$

(13)

$f_i^a$ is activated using the Sigmoid function to obtain channel-wise weights, which are element-wise multiplied with $f_i^{en}$ to produce the weighted feature. Formally,

$$f_i^{ca}=f_i^{en}\otimes \sigma \left(f_i^a\right)$$

(14)

Similarly, CBAM applies the spatial attention to the input $f_i^{ca}$. In particular, $f_i^{ca}$ undergoes max pooling and average pooling along the spatial dimension to generate two spatial context descriptors, and concatenates them as $f_i^{cat}$:

$$f_i^{cat}=\mathrm{Concate}\left(\mathrm{MaxPool}\left(f_i^{\mathrm{ca}}\right),\mathrm{AvgPool}\left(f_i^{ca}\right)\right)$$

(15)

The spatial weight is obtained by processing $f_i^{cat}$ through a $7\times 7$ convolution layer followed by a Sigmoid function. Then, this weight is multiplied in elements with $f_i^{ca}$ to generate the refined feature $f_i^e$.

$$f_i^e=\sigma \left({\mathrm{Conv}}_{7\times 7}\left(f_i^{cat}\right)\right)\otimes f_i^{ca}$$

(16)

3.4. Loss Function

The objective functions used to train our proposed DEPANet are a combination of binary cross-entropy loss (BCE) [25] and Dice loss (Dice) [26]. Formally

$$\mathcal{L}={\mathcal{L}}_{BCE}+{\mathcal{L}}_{Dice}$$

(17)

BCE loss is widely applied in binary classification tasks and is defined as

$${\mathcal{L}}_{BCE}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left|y_{i}log\left(p_i\right)+(1-y_i)log(1-p_i)\right|$$

(18)

where N represents the number of samples. $y_i$ is the true label of the sample i, where $y_i\in \{0,1\}$. The predicted probability for the positive class (i.e., $y_i=1$) is given by $p_i$.

The Dice loss is used for evaluating segmentation tasks and measures the similarity between two samples. It is given by

$${\mathcal{L}}_{Dice}=1-{\displaystyle \frac{2{\sum }_{i=1}^N{y}_i\widehat{y_i}}{{\sum }_{i=1}^N{y}_i+{\sum }_{i=1}^N\widehat{y_i}}}$$

(19)

where $y_i$ denotes the true label, $\widehat{y_i}$ is the predicted value for the sample i, and N is the total number of samples.

4. Experiment and Results

4.1. Datasets

To assess the effectiveness of DEPANet, experimental evaluations are conducted on the SD-saliency-900 dataset [27], a widely used benchmark for surface defect segmentation in strip steel manufacturing. This dataset contains 900 annotated samples with pixel-accurate $200\times 200$ masks, partitioned into 540 training images and 360 test images. As illustrated in Figure 4, numerous instances of various defect categories are presented. During training, input images are resized to $256\times 256$ pixels and standardized using Z-score normalization. During testing, the model processes $256\times 256$ resized inputs, with the generated segmentation output upscaled to the original $200\times 200$ resolution using bilinear interpolation for quantitative evaluation. This preprocessing pipeline maintains dimensional consistency while preserving defect characteristics, ensuring accurate performance evaluation.

4.2. Experimental Setup

We implement DEPANet using PyTorch version 2.3.1 and conduct experiments on a system equipped with an NVIDIA A100 80 GB PCIe GPU. The backbone is initialized with the parameters of ResNet34 [11] pretrained on ImageNet, while the remaining parameters are initialized using default settings. Adam is selected as the optimizer, with an initial learning rate of $lr=0.001$ and a batch size of 16. The training process is completed over 600 epochs.

4.3. Evaluation Metrics

Nine quantitative metrics are employed to assess the effectiveness of our DEPANet, including MAE [28], S-measure (SM) [29], weighted F-measure (w-$F_{\beta }$), Adaptive F-measure ($F_{\epsilon }$), Mean F-measure ($F_{\beta }$), Maximum F-measure ($F_{max}$), [30], Adaptive E-measure ($E_{\epsilon }$), Mean E-measure ($E_{\beta }$), and Maximum E-measure ($E_{max}$) [31].

The MAE quantifies pixel-level discrepancies through absolute differences between the saliency map S and binary annotation T:

$$\mathrm{MAE}={\displaystyle \frac{1}{H\times W}}\sum _{x=1}^H\sum _{y=1}^W\left|S\left(x,y\right)-T\left(x,y\right)\right|$$

(20)

where H and W denote spatial dimensions. The SM evaluates structural preservation through dual components:

$$\mathrm{SM}=\gamma \times Q_{obj}+\left(1-\gamma \right)\times Q_{reg}$$

(21)

where $Q_{obj}$ measures object-level shape fidelity, $Q_{reg}$ assesses regional coherence, and $\gamma$ balances their contributions.

The Weighted F-measure, Maximum F-measure, Mean F-measure, and Adaptive F-measure collectively provide a nuanced evaluation of precision and recall in the outcomes of saliency segmentation. These metrics are defined as follows:

$$F_{\gamma }={\displaystyle \frac{\left(1+{\gamma }^2\right)\times \mathcal{P}\times \mathcal{R}}{{\gamma }^2\times \mathcal{P}+\mathcal{R}}}$$

(22)

where $\mathcal{P}$ and $\mathcal{R}$ are computed with threshold-specific strategies. Different values of $\gamma$ can correspond to various versions of the F-measure, including the weighted F-measure, Maximum F-measure, Mean F-measure, and Adaptive F-measure.

Adaptive E-measure, Mean E-measure, and Maximum E-measure are metrics used to assess the similarity or matching degree between the predicted saliency map and the ground truth. The formulas are presented as follows:

$$E_{\xi }={\displaystyle \frac{1}{H\times W}}\sum _{x=1}^H\sum _{y=1}^W\psi \odot M\left(x,y\right)$$

(23)

where $\psi$ denotes an enhancement operator for local contrast adjustment, and $\xi$ determines evaluation mode: The Mean-E-measure averages across thresholds, Maximum E-measure selects optimal thresholds, and Adaptive E-measure uses adaptive thresholding.

4.4. Comparison Results

This section presents a comparative analysis of our DEPANet against 16 other methods on the SD-saliency-900 dataset, i.e., 2LSG [32], BASNet [33], BC [34], BMPM [35], CPD [36], DACNet [37], DSS [38], EDRNet [39], F3Net [40], ITSD [41], MIL [42], MINet [43], NLDF [44], PFANet [45], PiCANet [46], PoolNet [47], R3Net [48], RCRR [49], SAMNet [50], and SMD [51]. For a fair evaluation, we run the original code of the authors or reference their published experimental data.

Table 1. Performance comparison of competing methods, with top results highlighted in bold.

Methods	MAE ↓	SM ↑	w-${\mathit{F}}_{\mathit{\beta }}$ ↑	${\mathit{E}}_{\mathit{\epsilon }}$ ↑	${\mathit{E}}_{\mathit{\beta }}$ ↑	${\mathit{E}}_{\mathbf{max}}$ ↑	${\mathit{F}}_{\mathit{\epsilon }}$ ↑	${\mathit{F}}_{\mathit{\beta }}$ ↑	${\mathit{F}}_{\mathbf{max}}$ ↑	Params (M) ↓	FPS↑
2LSG [32]	0.2547	0.5723	0.2816	0.6745	0.5439	0.6572	0.4365	0.3954	0.4562	61.7	20.2
BASNet [33]	0.0159	0.9193	0.8955	0.9593	0.9603	0.9714	0.8694	0.9011	0.9223	52.1	23.1
BC [34]	0.1593	0.5861	0.3999	0.7264	0.6403	0.6836	0.4677	0.4227	0.4708	——	18.0
BMPM [35]	0.0312	0.8513	0.8454	0.9298	0.8972	0.9301	0.8384	0.8417	0.8622	49.9	26.2
CPD [36]	0.0242	0.8981	0.7928	0.9299	0.9369	0.9744	0.8305	0.8701	0.9091	29.6	19.2
DACNet [37]	0.0129	0.9321	0.9139	0.9671	0.9693	0.9627	0.8911	0.9217	0.9351	98.2	15.7
DSS [38]	0.0256	0.8202	0.8296	0.9339	0.8514	0.9326	0.8581	0.8033	0.8501	74.9	24.6
EDRNet [39]	0.0139	0.9343	0.9187	0.9635	0.9716	0.9790	0.8858	0.9177	0.9370	49.4	27.9
F3Net [40]	0.0157	0.9155	0.9065	0.9680	0.9671	0.9724	0.8943	0.9038	0.9178	35.8	23.5
ITSD [41]	0.0158	0.9201	0.8976	0.9561	0.9641	0.9733	0.8606	0.8923	0.9207	48.1	18.0
MIL [42]	0.1854	0.6143	0.3087	0.7216	0.5971	0.7079	0.4760	0.4472	0.5385	56.5	20.8
MINet [43]	0.0150	0.9202	0.9181	0.9604	0.9631	0.9701	0.8813	0.9011	0.8979	48.3	19.6
NLDF [44]	0.0491	0.8021	0.6911	0.671	0.8050	0.8734	0.7211	0.7007	0.7621	56.3	23.4
PFANet [45]	0.0876	0.7381	0.5322	0.7195	0.7500	0.8549	0.5545	0.5922	0.7089	88.0	17.6
PiCANet [46]	0.0289	0.8912	0.8902	0.8961	0.9199	0.9659	0.7580	0.8278	0.8022	49.1	26.7
PoolNet [47]	0.0244	0.9005	0.8322	0.9173	0.9311	0.9692	0.8062	0.8484	0.8905	54.4	25.0
R3Net [48]	0.0263	0.8374	0.8301	0.9298	0.9022	0.9341	0.8419	0.8280	0.8351	48.3	27.0
RCRR [49]	0.2455	0.5311	0.2317	0.6345	0.5465	0.6261	0.3327	0.3381	0.3923	——	20.0
SAMNet [50]	0.0267	0.9013	0.8491	0.9270	0.9385	0.9700	0.8045	0.8577	0.9027	65.5	29.3
SMD [51]	0.2088	0.5808	0.3652	0.7093	0.5921	0.6467	0.4411	0.4233	0.4612	74.1	19.1
Ours	0.0132	0.9373	0.9247	0.9793	0.9765	0.9793	0.9223	0.9193	0.9256	47.6	22.3

compares the methods in the SD-saliency-900 dataset, and Figure 5 visualizes their F-measure and Precision–Recall curves.

From Table 1, our DEPANet demonstrates significant advantages across multiple metrics, outperforming other comparative methods in overall performance. Firstly, our DEPANet ranks first in terms of structural similarity (SM, 0.9373) and weighted F-measure ($w-F_{\beta }$, 0.9347), indicating its exceptional ability to capture image structures and details. Secondly, our method also achieves the best results in metrics related to edge detection ($E_{\epsilon }$, 0.9793; $E_{\beta }$, 0.9765; $E_{max}$, 0.9793), further proving its superior capability in edge feature extraction and enhancement. Furthermore, our method reaches the highest level in F-measure ($F_{\epsilon }$, 0.9223; $F_{\beta }$, 0.9193) and ties with EDRNet in $F_{max}$ (0.9370), demonstrating its strong competitiveness in comprehensive evaluation metrics. In terms of deployment feasibility, DEPANet has 47.6 million parameters, significantly fewer than models like DACNet (98.2M), which makes it more memory-efficient and suitable for embedded GPU or edge-device deployment.

In addition to segmentation accuracy, inference time is a critical factor for practical deployment in steel production lines, where real-time or near real-time processing is often required. As shown in Table 1, DEPANet achieves a 22.3 FPS on an NVIDIA A100 GPU with a batch size of 16 for $256\times 256$ images.

Figure 5 shows that DEPANet outperforms all other models, demonstrating a high F-measure at different thresholds and superior precision at various recall rates. The DEPANet precision recall curve is significantly closer to the ideal (1,1) point compared to other models, while its F-measure curve consistently dominates those of alternative approaches. Furthermore, when evaluated against state-of-the-art models, including PiCANet, DACNet, R3Net, and MINet, DEPANet maintains a significant performance advantage. These comprehensive experimental results validate both the effectiveness and reliability of the proposed DEPANet architecture.

4.5. Visualization Analysis

Figure 6 visually compares our DEPANet with competing methods in diverse images. The third column displays our DEPANet segmentation results. Figure 6 shows that DEPANet consistently delivers the highest accuracy in defect segmentation, enabling complete and precise delineation of defect areas. Our method effectively detects and segments defects across a range of sizes and morphological variations, demonstrating robustness to both small and large defects. In the images shown in Rows 1–3, where the contrast between the defect regions and the background is low, existing methods exhibit two typical failure modes. Methods such as BASNet and MINet, being overly sensitive to texture variations, often misclassify background regions as defects. In contrast, methods such as 2LSG and BC, due to their limited feature representation capability, frequently fail to fully segment the defect regions. For images in Rows 4–6, which contain finely detailed defects, certain models yield segmentation results with unclear boundaries. However, DEPANet produces results with sharp and well-defined contours, ensuring precise defect localization. In the last two rows of images, which contain complex backgrounds and challenging scenarios, most existing methods generate saliency maps with poorly defined edges and imprecise highlighting of defect regions. In contrast, DEPANet effectively distinguishes defect regions from the background, demonstrating superior segmentation capability.

To further illustrate the role of edge information extracted by DEFP, we visualize the intermediate edge maps generated at different stages of DEFP in Figure 7. As shown in Figure 7, Stage 1 and Stage 2 focus on capturing fine-grained textures and noise-like contours, while other stages highlight more semantically meaningful and structurally consistent defect boundaries. These progressive representations demonstrate the hierarchical edge learning ability of our AFSFs and provide insight into how DEPANet progressively refines edge information to assist defect segmentation.

4.6. Ablation Studies

We conduct ablation studies to evaluate the impact of key components, including the ResNet34 backbone, DEFP, EFAM, and CBAM modules, as well as the BCE and Dice loss functions, using evaluation metrics of MAE, SM, $E_{\beta }$, and $F_{\beta }$.

Table 2. The experimental results of ablation studies conducted using the dataset SD-saliency-900.

Exp.	DEPANet				SD-Saliency-900
	ResNet34	DEFP	EFAM	CBAM	MAE ↓	SM ↑	${\mathit{E}}_{\mathbf{max}}$ ↑	${\mathit{F}}_{\mathbf{max}}$ ↑
#1	✓	X	X	X	0.0191	0.8891	0.9231	0.8919
#2	✓	✓	✓	X	0.0141	0.9267	0.9671	0.9298
#3	✓	✓	✓	✓	0.0132	0.9373	0.9758	0.9396
#4	BCE				0.0154	0.9015	0.9598	0.9116
#5	Dice				0.0149	0.9141	0.9674	0.9180
#6	BCE + Dice				0.0132	0.9373	0.9758	0.9396

(Rows #1–#3) shows the impact of key components, including the ResNet34 backbone, DEFP, EFAM, and CBAM modules, where Row #1 is used as the baseline. As evidenced in Table 2 (Rows #1–#3), the model exhibits a progressive and substantial performance enhancement with sequential incorporation of each component. Specifically, the model combining DEFP and EFAM (Row #2) achieves improved performance compared to Row #1, demonstrating the complementary effectiveness of the edge-aware mechanism in enhancing feature attention. Finally, Row #3, which integrates all modules (DEFP, EFAM, and CBAM), achieves the best performance with the lowest MAE (0.0132) and highest SM (0.9373), $E_{max}$ (0.9758), and $F_{max}$ (0.9396), conclusively proving the synergistic effectiveness of these components in enhancing the overall model performance.

Table 2 (Rows #4–#6) shows the impact of the BCE and Dice loss functions on our DEPANet. From Table 2 (row #6), our DEPANet trained with hybrid loss, i.e., BCE and Dice, achieves improvements of 0.22% (0.17%), 3.58% (2.32%), 1.6% (0.84%), and 2.4% (1.76%) over the model trained with BCE (Dice) in terms of MAE, SM, $E_{\beta }$, and $F_{\beta }$, respectively.

4.7. Generalization Experiments

While the SD-saliency-900 dataset provides valuable insight with its three steel surface defect categories, its limited scope may not fully represent all possible defect variations. Potential biases in data collection, particularly from specific production environments, could also impact model generalization across diverse industrial applications.

To comprehensively evaluate the DEPANet’s performance, we performed additional tests on the ESDIs-SOD dataset [52]. Following established protocols, we trained the model on 3600 images (1200 per class) and tested on the remaining 1200 images.

Table 3. DEPANet is comprehensively evaluated against 10 state-of-the-art methods on the ESDIs-SOD dataset, with optimal results highlighted in bold.

Methods	MAE ↓	SM ↑	${\mathit{E}}_{\mathbf{max}}$ ↑	${\mathit{F}}_{\mathbf{max}}$ ↑	Parameters (M) ↓
C2FNet [53]	0.0207	0.8982	0.9701	0.8835	26.4
CPD [36]	0.0254	0.8884	0.9627	0.8777	29.6
DACNet [37]	0.0226	0.8855	0.9570	0.8722	98.2
EDRNet [39]	0.0229	0.8651	0.9603	0.8769	49.4
BC [34]	0.1356	0.6004	0.6704	0.4976	—
F3Net [40]	0.0208	0.8977	0.9663	0.8910	35.8
MINet [43]	0.0221	0.8885	0.9604	0.8798	48.3
RCRR [49]	0.2194	0.5633	0.6198	0.4377	—
CorrNet [54]	0.0243	0.8776	0.9532	0.8785	39.3
PiCANet [46]	0.0449	0.8490	0.9317	0.8337	49.1
Ours	0.0202	0.9007	0.9654	0.8860	47.6

presents a comparative evaluation of DEPANet against 10 methods. The results demonstrate superior performance across all key metrics (MAE, SM, $E_{max}$, $F_{max}$, and parameter efficiency), confirming the method’s robustness and effectiveness.

5. Conclusions

In this paper, we propose DEPANet, an encoder–decoder architecture for strip steel surface defect segmentation. The encoder consists of three key components: a backbone, a Differentiable Edge Feature Pyramid (DEFP) network, and Edge-aware Feature Aggregation Modules (EFAMs). The backbone utilizes ResNet34 to extract comprehensive features from the strip steel surface. The DEFP network extracts multi-scale edge features through a series of Adaptive Feature Smoothing Filters (AFSFs). Each AFSF incorporates two learnable components: a Laplacian operator for edge feature extraction and a Gaussian blur mechanism for feature smoothing and enhancement. The multiple proposed EFAMs are employed to generate edge-enhanced features, with each EFAM designed to integrate the global backbone features with the edge components from DEFP at the corresponding level. In addition, we incorporate a Convolutional Block Attention Module (CBAM) to ensure effective fusion of global context and fine edge detail. Experimental results demonstrate that DEPANet effectively enhances defect segmentation accuracy in complex industrial environments.

In future work, we will focus on two key improvements: (1) developing a lightweight variant of DEPANet for real-time defect detection in resource-constrained industrial settings and (2) extending our framework to handle temporal information from production line videos to better capture transient defect patterns.