Improved SegFormer with Guided Multi-Scale Fusion and Boundary-Aware Attention for Slippery Road Recognition

Highlights

What are the main findings?

• The proposed Boundary-Aware Attention (BAA) module with directional Sobel-based feature pooling and the Guided Multi-scale Fusion (GMF) module significantly improve boundary delineation and multi-scale feature integration for slippery road recognition.
• A multi-scale input strategy with a compound loss function (Cross-Entropy & Dice Loss) achieves 95.80% mIoU and 97.84% accuracy on a real-world highway dataset, effectively mitigating class imbalance and low-contrast issues.

What are the implications of the main findings?

• The method maintains strong performance on the Cityscapes benchmark (95.80% mIoU, 76.20% AmIoU), demonstrating excellent cross-scene generalization beyond highways.
• This work provides a reliable technical foundation for highway slippery road warning systems, with high potential for real-world deployment in diverse weather and traffic conditions.

Abstract

Accurate and timely identification of slippery road surfaces is essential for ensuring driving safety and operational efficiency on highways. However, blurred vehicle-background boundaries, uneven illumination, and water splashing caused by passing vehicles make existing image-based recognition methods prone to low accuracy. To address these challenges, this paper proposes an improved SegFormer-based framework with two task-specific innovations: (1) a novel Guided Multi-scale Fusion (GMF) module for task-guided multi-scale feature integration, designed to incorporate auxiliary information such as vehicle type, relative speed, and splash regions, enabling the network to focus on slipperiness-relevant road areas while suppressing background interference; and (2) an improved Boundary Attention Awareness (BAA) module with directional Sobel-based boundary initialization, which provides explicit geometric priors to preserve fine boundary details and reduce ambiguity in slippery regions with irregular or weak edges. A multi-scale input and enhancement strategy is further adopted, along with a weighted combination of cross-entropy loss and Dice loss to mitigate class imbalance. Experimental results on our self-constructed Guangzhou Beierhuan Expressway dataset achieve an mIoU of 95.80%, accuracy of 97.84%, and F1-score of 97.86%. To verify cross-domain generalization, we further evaluate the model on two additional benchmarks: it achieves an mIoU of 93.51% on the synthetic SYN-UDTIRI dataset, and attains an mIoU of 95.80% with an AmIoU of 76.20% on the public Cityscapes dataset, achieving competitive performance against several mainstream architectures. The proposed method offers considerable application potential for highway safety warning systems.

1. Introduction

Slippery road conditions significantly threaten highway safety by reducing tire–pavement adhesion, leading to longer braking distances, poorer handling, and increased skidding or loss of control [1]. Globally, road traffic accidents cause about 1.19 million deaths annually [2]. Highways are especially vulnerable to rain, ice, and water accumulation, which can induce skidding or overturning [3]. Real-time recognition of wet, snowy, or icy surfaces is therefore critical for highway authorities to implement proactive strategies like variable speed limits and timely maintenance [4,5]. With widespread video surveillance, image-based methods offer continuous, non-contact monitoring without costly embedded sensors [6].

Early research predominantly employed traditional machine learning, applying support vector machine (SVM) classifiers [7] or BP neural networks [8] with hand-crafted features. However, these methods rely on manually designed features that capture only superficial information, leading to limited generalization and poor robustness under varying illumination, making them unsuitable for complex multi-scale backgrounds.

With the advancement of deep learning, convolutional neural networks (CNNs) have been increasingly adopted for their ability to automatically learn hierarchical features. Gui et al. [9] proposed a lightweight RefNet for low-power devices, while Cai et al. [10] introduced HSA-ResNet50 with hybrid attention. Wang et al. [11] used FC-DenseNet56 for road type classification, and Li et al. [12] proposed an improved SegNet with pyramid pooling. Recent advances in deep learning-based semantic segmentation have further demonstrated the potential of these techniques for structural inspection and road-condition assessment across various domains [13,14,15,16,17]. Specialized segmentation networks have also been developed for pavement crack detection, employing dynamic dual-branch encoders with deformable spatial focusing to better capture irregular crack morphologies [18], or utilizing dual-branch transitional feature coupling and hierarchical edge-guided fusion to suppress complex background interference [19]. For railway scene understanding, structure-aware training strategies have been introduced to preserve the continuity of thin elongated rail structures while maintaining real-time efficiency [20]. However, these CNN-based methods, despite achieving high accuracy in some cases, still struggle with highly variable illumination, indistinct boundaries between wet roads and surroundings, and ineffective multi-scale feature fusion, resulting in insufficient accuracy and weak generalization when faced with small distant water patches or strong light interference [21].

More recently, Transformer-based architectures have demonstrated remarkable capabilities in capturing long-range dependencies for semantic segmentation tasks [22,23,24,25]. The SegFormer framework [22] combines a hierarchical Transformer encoder with a lightweight MLP decoder, balancing accuracy and efficiency. However, its global attention suffers from feature confusion on occluded and blurred boundaries, and the lack of explicit semantic guidance limits its adaptability to complex traffic scenes. Deep learning methods for image recognition under difficult weather conditions [26] also highlight the persistent challenges in achieving robust performance in real-world driving environments. Consequently, existing methods still suffer from insufficient accuracy for the specific challenges of highway wet-surface recognition.

To address these limitations, several improvement strategies have been proposed. Luo et al. [27] integrated squeeze-and-excitation modules and an auxiliary edge-fusion branch for landslide recognition. Zhang et al. [28] proposed MF-SegFormer for water body extraction, integrating multi-scale fusion and ASPP modules. Boundary-aware attention mechanisms [29,30] and multi-scale fusion strategies [31] have proven effective in medical and remote sensing segmentation. Further improvements to SegFormer have also been explored for other specialized scenarios [32,33], demonstrating the flexibility of the architecture. Despite these advances, applying these techniques directly to slippery road recognition remains difficult due to the unique challenges posed by varying illumination, vehicle speed, and complex backgrounds.

Inspired by these advances, this paper proposes an improved SegFormer-based method for slippery road surface recognition on highways. We integrate a Boundary-Aware Attention (BAA) module [29] to delineate precise road boundaries under challenging illumination, and a Guided Multi-scale Feature Fusion (GMF) mechanism [34] that incorporates vehicle semantic information—such as vehicle type, relative speed, and splash region—to construct an attention weight map for optimizing local key region features. To further enhance the performance of these modules for the specific task of slippery road recognition, we introduce two key improvements: (1) a novel Guided Multi-scale Fusion (GMF) module for task-guided multi-scale feature integration, and (2) an improved Boundary Attention Awareness (BAA) module with directional Sobel-based boundary initialization to enhance boundary localization. A multi-scale input strategy and a compound loss function (focal, dice, and boundary loss) are further adopted. The proposed approach enables multi-level classification of dry, wet, and water-covered conditions. The method is validated on a newly collected dataset from the Guangzhou Beierhuan Expressway, which serves as one of the primary data sources for this study. This expressway, with a total length of approximately 38.5 km, was opened to traffic in January 2002 and, as one of the major transportation corridors in the Pearl River Delta, has experienced a continuous increase in traffic demand driven by rapid regional socioeconomic development.

The remainder of this paper is structured as follows. Section 2 details the methodologies, including the limitations of standard SegFormer and the proposed improvements. Section 3 describes dataset construction and evaluation metrics. Section 4 presents experimental results and comparisons. Section 5 discusses model characteristics and potential shortcomings. Main conclusions are presented in Section 6.

2. Methodology

It is worth noting that the visual slipperiness recognized by computer-vision models is fundamentally different from actual physical slipperiness. Physical slipperiness is determined by the real-time tire–pavement friction coefficient, which is influenced by pavement macrotexture and microtexture, water-film thickness, vehicle speed, tire inflation pressure, and tire tread wear. The camera-based framework proposed in this study primarily identifies visual road-surface conditions, including dry, wet, and puddled states, together with explicit splash characteristics. These visual cues provide early indicators of hazardous road conditions and serve as essential boundary inputs for subsequent physical risk-assessment models, including hydroplaning prediction and braking-distance estimation.

Regarding the literature review, we have restructured the Introduction to follow the recommended order: road-condition detection, semantic segmentation, and finally SegFormer, attention mechanisms, and multi-scale fusion approaches. In this section, the overall framework of the improved SegFormer-based method is described. Main characteristics of pavement slipperiness and its determination are introduced first, along with shortcomings of applying SegFormer on pavement slipperiness image recognition. To improve the precision and effectiveness of pavement slipperiness determination, the improved SegFormer-based method is proposed, and its main properties are introduced secondly. After that, the Guided Multi-Scale Fusion Module (GMF) [34], Boundary-Aware Attention Module (BAA) [29], and Multi-scale Input and Enhancement Strategy are presented to outline the main advantages of the improved SegFormer-based method.

2.1. Pavement Slipperiness Determination and SegFormer

When the surface of highway pavement becomes wet or water-covered, its adhesion coefficient becomes significantly lower than that of dry pavement, which impairs vehicle handling stability and compromises driving safety. Compared with dry pavement images, images of wet or puddled pavements exhibit distinct visual characteristics due to the water film covering the road surface, as well as the splash traces left by passing vehicles. On the one hand, the reflective and mirror-like effects of the water film introduce substantial differences between wet and dry pavement images. On the other hand, when vehicles traverse wet or puddled sections, the increasing dynamic water pressure may lift the wheels off the road surface, leading to hydroplaning and leaving visible splash traces. Consequently, the slipperiness state of the pavement can be inferred from camera-captured images. Based on the degree of water accumulation, pavement conditions are classified into three categories: dry, wet, and puddled.

SegFormer is a fine-grained artificial neural network designed for efficient and precise image semantic segmentation. Its main framework is illustrated in Figure 1. As shown in Figure 1, the encoder and decoder are the two core components. The encoder adopts a Mix Transformer (MiT) structure, which first decomposes input images into smaller patches via overlapping patch embeddings, and then progressively extracts multi-scale features through four Transformer blocks. These blocks integrate efficient self-attention mechanisms and a Mix Feed-Forward Network (Mix-FFN). After processing, the sizes of the resulting feature maps are reduced to approximately 1/4, 1/8, 1/16, and 1/32 of the original input resolution, respectively. Subsequently, the multi-scale feature maps are fed into the decoder, which is constructed with Multilayer Perceptrons (MLPs). Through MLP-based sampling, low-resolution feature maps are upsampled and fused with features from different scales, enabling the SegFormer model to capture comprehensive contextual information.

Although SegFormer can effectively capture the main features of images, its recognition precision declines when processing wet or puddled pavement images, largely because the boundary between water traces and the surrounding environment becomes blurred under insufficient lighting. Moreover, relying solely on the SegFormer model is inadequate for precise slipperiness recognition, given the complex coupling among vehicles, pavement surfaces, and water in the image. Therefore, developing an improved SegFormer model to overcome these limitations is critically important.

2.2. Framework of the Improved SegFormer

Standard SegFormer has two shortcomings for slippery pavement recognition: (1) insufficient capability to recognize small-sized slippery regions; and (2) difficulty in locating slippery boundaries under varying illumination. To address these issues, an improved SegFormer-based framework is proposed.

The original SegFormer decoder performs simple MLP-based fusion of multi-scale features without guidance for salient pavement regions, leading to insufficient performance in fine-grained detail regions such as wheel-near water accumulation, water traces, and road reflections. To overcome this, we introduce a Guided Multi-scale Fusion (GMF) module at the decoder stage, integrating a vehicle semantic guidance branch prior to feature fusion. This branch constructs an attention weight map using external auxiliary cues—vehicle category, relative speed level, and splash region locations—to adaptively recalibrate multi-scale feature responses, effectively suppressing background interference such as roadside vegetation and guardrail shadows while emphasizing task-relevant regions [34].

To address blurred boundaries of water-accumulation regions and their confusion with shadows or specular reflections, a Boundary Attention Awareness (BAA) module [29] is inserted between the encoder output and decoder input. Unlike conventional omnidirectional edge extraction, the BAA module employs directional Sobel-based feature pooling to capture horizontally extended water films and continuous splash patterns. The directionally pooled edge map generates a boundary attention map fused with intermediate semantic features, strengthening boundary contour representation and reducing label drift.

By incorporating GMF and BAA, the standard SegFormer is optimized for highway slippery condition detection. The overall architecture is illustrated in Figure 2. Additionally, a multi-scale input and enhancement strategy is adopted, and a combined Cross-Entropy and Dice loss is employed to balance class imbalance and boundary accuracy.

2.3. Guided Multi-Scale Fusion and Boundary-Aware Attention Module

The core idea of the GMF module [34] lies in introducing a guidance mechanism derived from vehicle semantic information prior to feature fusion in the decoding stage, thereby enhancing the model’s ability to focus on task-critical regions. As for the BAA module, it resides in leveraging edge information as guidance, extracting prominent boundary features from the input image through edge operators.

2.3.1. The GMF Module

The GMF module [34] introduces a guidance mechanism derived from vehicle semantic information prior to feature fusion in the decoding stage. It extracts prior features including vehicle type, relative speed, and splash region location, and applies dynamic attention adjustment weights to semantic feature maps of various scales. The mechanism of the GMF module [34] is presented in Figure 3. Through this guidance mechanism, the model prioritizes enhancing regions highly relevant to slippery conditions during multi-scale fusion. The core operations are formulated as

$${\mathit{F}}_i^{\prime }={\mathit{A}}_i\odot {\mathit{F}}_i$$

(1)

$${\mathit{A}}_i=\mathsf{\sigma }\left({\mathrm{Conv}}_1\left(\mathrm{ReLU}\left({\mathrm{Conv}}_2\left(\mathit{G}\right)\right)\right)\right)\hspace{0.17em}$$

(2)

where ${\mathit{F}}_i$ is the original feature map for the ith layer scale and its dimension is $C_i\times H_i\times W_i$. ${\mathit{A}}_i$ denotes the weight map of guided attention, which has the same dimension as ${\mathit{F}}_i$. $\odot$ represents the pixel-wise multiplication operation. $\mathit{G}$ denotes the multi-modal prior embedding encompassing vehicle dynamics and spatial location of vehicle information, such as vehicle category, speed level, and splash zone location, etc. The weight map of guided attention ${\mathit{A}}_i$ is generated after two successive convolutional layers. $Con{v}_1$ and $Con{v}_2$ represent convolution layers. $\mathsf{\sigma }$ denotes the $Sigmoid$ activation function.

The output function after multi-scale fusion can be expressed as Equation (3),

$$F_{out}=\mathrm{Concat}({\mathsf{\varphi }}_1(F_{f,1}),{\mathsf{\varphi }}_2(F_{f,2}),\dots ,{\mathsf{\varphi }}_n(F_{f,n}))$$

(3)

where $F_{out}\hspace{0.17em}$ is the input feature map of the Decoder. $F_{f,j}$ represents the guided feature map of the $j\text{-th}$ scale. ${\mathsf{\varphi }}_i(\cdot )\hspace{0.17em}$ denotes a spatial alignment transformation consisting of a $1\times 1$ convolution for channel adjustment and bilinear interpolation for resolution up-sampling. Finally, $\mathrm{Concat}$ represents the channel-wise concatenation that integrates multi-scale semantic information for final road state recognition.

Conventional self-attention computes pairwise similarities over all spatial locations, which may allocate considerable attention to irrelevant background regions. In contrast, the proposed GMF introduces task-oriented guidance to constrain the attention distribution toward semantically meaningful regions associated with slippery pavement. This guided feature recalibration reduces intra-class variance while enlarging inter-class feature separability, leading to more discriminative multi-scale representations.

2.3.2. The BAA Module

The core of the Boundary-Aware Attention Module (the BAA module, whose key principle is illustrated in Figure 4) [29] resides in leveraging edge information as guidance, extracting prominent boundary features from the input image through edge operators. Subsequently, these edge maps are utilized to generate boundary-aware attention maps, guiding the model to more accurately focus on regional boundaries in feature maps. This mechanism significantly enhances the model’s responsiveness to the edge contours of slippery regions, effectively mitigating boundary ambiguity and mis-segmentation at the edges of water accumulations.

CNN-based encoders inevitably lose high-frequency boundary information after repeated downsampling. The proposed BAA explicitly injects edge priors through directional Sobel initialization, providing geometric constraints during feature fusion. Consequently, boundary responses are preserved throughout the decoding process, reducing boundary uncertainty and improving localization accuracy for thin water films and splash regions.

The generation of the boundary attention map in the BAA Module [29] is presented in Equation (4),

$${\mathit{M}}_{edge}=\mathsf{\sigma }(\mathrm{Conv}(\text{Sobel}(\mathit{I})))$$

(4)

where $\mathit{I}$ is the original input image and $\mathrm{Sobel}(\mathit{I})$ denotes the Sobel edge detection of the original input image $\mathit{I}$. ${\mathit{M}}_{edge}$ is the boundary attention map, whose value ranges from 0 to 1.0.

The boundary-enhanced feature map can be obtained as Equation (5),

$${\mathit{F}}_{\mathrm{final}}=\mathit{F}+{\mathit{M}}_{\text{edge}}\odot \mathit{F}$$

(5)

The boundary attention map can be enhanced with information around edges,

$${\mathit{F}}_{edge}^{\prime }={\mathit{M}}_{\mathrm{edge}}\odot \mathit{F}$$

(6)

where $\mathit{F}$ is the feature output of the current Transformer, and ${\mathit{F}}_{\mathrm{final}}$ the final boundary-enhanced feature map.

2.3.3. Multi-Scale Input and Enhancement Strategy

Considering that slippery pavement regions may appear in images merely as subtle texture variations or local brightness differences—particularly under uneven illumination, long-distance imaging, motion blur, or splashing conditions—traditional single-scale input models often struggle to accurately detect such areas. To enhance the model’s perceptual capability, this study proposes an improved method that integrates multi-scale image inputs with a heterogeneous feature fusion strategy.

First, during the training stage, input images are fed into different model layers at multiple scaling sizes to simultaneously capture global scene structures and local texture details. Image augmentation techniques such as Gaussian blur, synthetic raindrop texture overlay, and brightness perturbation are applied to improve model robustness under diverse environmental variations. Second, to further integrate features extracted from different scales and modalities, this study introduces a Heterogeneous Feature Fusion Module (HFFM) and a Feature Recalibration Module (FFRM). Detailed formulations are provided in Appendix A.

2.3.4. The Coupled Loss Function

During the model training process, a weighted combination of Cross-Entropy Loss and Dice Loss was chosen as the overall objective function, aiming to address both class imbalance and the accuracy of boundary region recognition. The total loss function is defined as:

$${\mathcal{L}}_{total}=\alpha \cdot {\mathcal{L}}_{\mathrm{CE}}+\left(1-\alpha \right)\cdot {\mathcal{L}}_{\mathrm{Dice}}$$

(7)

where ${\mathcal{L}}_{total}$ denotes the total loss during model training; α is the weighting coefficient between the Cross-Entropy Loss and Dice Loss; ${\mathcal{L}}_{\mathrm{CE}}$ represents the Cross-Entropy Loss; and ${\mathcal{L}}_{\mathrm{Dice}}$ denotes the Dice Loss, which is used to measure the overlap between segmented regions. Detailed formulations of ${\mathcal{L}}_{\mathrm{CE}}$ and ${\mathcal{L}}_{\mathrm{Dice}}$ are provided in Appendix A.

During the model optimization phase, the AdamW optimizer is employed. To balance training efficiency and performance, a mixed-precision training mechanism is introduced to accelerate computation and optimize memory utilization. Meanwhile, an Early Stopping strategy is applied to prevent overfitting, coupled with a dynamic class-weight adaptive adjustment mechanism to tackle data imbalance in underrepresented regions. The entire training process incorporates multi-round data augmentation and 5-fold cross-validation, ensuring stable convergence and robust boundary segmentation for subsequent real-world deployment.

3. Dataset and Performance Metrics

To verify the effectiveness of the proposed improved SegFormer-based pavement slipperiness recognition method, a dataset is constructed based on images collected from the Beierhuan expressway, Guangzhou. Details on the dataset are presented in this section. In addition, performance metrics like Pixel Accuracy (Acc), Mean Intersection over Union (mIoU), Average mean Intersection over Union (AmIoU) and F1 score are adopted to evaluate the performance of the proposed model. And the Frames Per Second (FPS) is also used in this article to assess the real-time processing capability.

3.1. Dataset

The raw image data were collected over a seven-month period from April to October 2024, fully covering the plum-rain season and typhoon season in southern China. The average daily traffic volume of the Guangzhou Beierhuan Expressway has reached approximately 130,000 vehicles, while the roadway has been operating at Level of Service (LOS) IV–V for extended periods, indicating frequent traffic congestion and heavy operational pressure. Such conditions not only reduce traffic efficiency but also increase the likelihood of traffic accidents, particularly under adverse weather conditions. According to traffic statistics, the expressway recorded an average of approximately 16 traffic accidents per day between 2016 and 2020.

The data were captured by fixed surveillance cameras deployed at five distinct representative locations along the Guangzhou Beierhuan Expressway. To maintain computational efficiency and consistency, all collected images were uniformly cropped and resized to a resolution of 1024 × 512 pixels. The dataset was manually annotated using strict semantic guidelines to minimize subjectivity. The initial labeling task was executed independently by three graduate students specializing in traffic and highway engineering. To guarantee annotation rigor and eliminate label drift, a senior domain expert conducted a secondary cross-check and formal quality-control evaluation on all annotated frames. Discrepancies in ambiguous boundary regions were discussed and resolved collectively to maintain high inter-annotator agreement.

In order to verify the capability of the proposed model comprehensively, we created a high-quality image dataset for semantic segmentation. In this paper, over 10,000 pavement images from the Guangzhou Beierhuan Expressway are used to create the dataset, which are mainly captured by fixed monitoring systems and roadside cameras. Those images inside the dataset were acquired in sunny, rainy, and post-rain weather conditions, with various lightness, vehicle density, and pavement conditions being considered and both scene diversity and representativeness being ensured. Typical images considering different weather conditions, vehicle density and slipperiness are presented in Figure 5.

Images inside the dataset are separated into a training set, a validation set and a test set with the ratio of 8:1:1, which can help us keep a balance on different slipperiness among the three subsets. Before the partition of subsets, images were labeled manually by using a pixel-wise semantic segmentation method. Visual characteristics of slipperiness were exploited to label images in the dataset into three classes: dry, wet, and puddled, due to whether there exist water traces, water films, and visibly reflective water accumulation. Non-pixel-level auxiliary labeling information like vehicle type, relative speed level, and spatial location features of splash zones are introduced to enhance the model’s focus on potentially slippery regions. That information serves as semantic priors during training to improve the model’s perception and discrimination capabilities in real-world highway scenarios. To eliminate subjectivity, strict labeling rubrics were applied: “Wet” specifies a darkened asphalt texture with low-intensity discrete reflection but without open water pooling; “Puddled” requires a continuous mirror-like water film accompanied by prominent specular reflections or continuous wheel-induced splash clouds.

Dataset Characteristics and Potential Bias. The proposed dataset mainly consists of highway pavement images collected under daytime conditions on the Guangzhou Beierhuan Expressway. Most road surfaces are dense-graded asphalt concrete, while extreme scenarios such as snow-covered pavements, icy roads, and severe nighttime glare are underrepresented. To ensure annotation quality, all images were manually labelled following unified annotation guidelines and subsequently verified through multiple rounds of inspection. Although the dataset covers diverse wet pavement conditions, its geographical and environmental diversity remains limited, which may affect the model’s generalization to substantially different domains. The Numbers and percentages of images that are labelled as Dry, Wet, and Water are presented in

Table 1. Class distribution of the proposed road surface dataset.

Class	Number of Images	Percentage
Dry	8756	38.9%
Wet	7543	33.5%
Water	6201	27.6%

.

3.2. Quantitative Metrics

The calculation of Pixel Accuracy (Acc) can be expressed as,

$$Acc={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(8)

where $Acc$ represents the pixel accuracy. $TP$, $TN$, $FP$ and $FN$ denote the true positive examples, true negative examples, false positive and false negative examples, respectively.

The Mean Intersection over Union (mIoU) is taken as,

$$mIoU={\displaystyle \frac{Io{U}_1+Io{U}_2+\cdots +Io{U}_n}{n}}$$

(9)

In which, $Io{U}_1$, $Io{U}_2$, … $Io{U}_n$ (n = 3 in this paper to consider wet, dry and puddled slipperiness conditions) are the Intersection over Unions and it can be calculated by $IoU=TP/\left(TP+FP+FN\right)$.

Average mean Intersection over Union (AmIoU) is defined as,

$$AmIoU={\displaystyle \frac{mIo{U}_1+mIo{U}_2+\cdots +mIo{U}_k}{k}}$$

(10)

where k represents the number of scenes in the dataset, considering various combinations of different road sections and weather conditions.

Additionally, the F1-score is calculated to balance Precision and Recall:

$$AmIoU={\displaystyle \frac{mIo{U}_1+mIo{U}_2+\cdots +mIo{U}_k}{k}}$$

(11)

where $\mathrm{Recall}=TP/\left(TP+FN\right)$.

4. Results and Analysis

Multiple experiments are conducted to validate the effectiveness of the proposed improved SegFormer-based method, including validation of the effectiveness of the BAA and GMF module, comparisons with 9 classical semantic segmentation models, tests on different datasets, and a visualization analysis case.

4.1. Validation on the Effectiveness of BAA and GMF Module

To validate the effectiveness of GMF and BAA, experiments are conducted with different module combinations. As shown in

Table 2. Ablation studies on the effectiveness of BAA and GMF modules.

BAA	GMF	mIoU (%)	Acc (%)	F1 Score (%)	Precision (%)	Recall (%)	FPS
×	×	94.11 ± 0.21	95.87 ± 0.17	97.50 ± 0.15	98.12 ± 0.18	96.87 ± 0.20	21.80
✓	×	95.45 ± 0.16	97.63 ± 0.13	97.67 ± 0.16	97.95 ± 0.13	97.40 ± 0.17	21.60
×	✓	95.42 ± 0.19	97.59 ± 0.14	97.69 ± 0.13	97.79 ± 0.17	97.59 ± 0.15	21.60
✓	✓	95.80 ± 0.14	97.84 ± 0.08	97.86 ± 0.09	97.74 ± 0.11	97.97 ± 0.12	20.30

, the baseline SegFormer achieves an mIoU of 94.11%. Incorporating BAA alone improves mIoU to 95.45%, with enhanced boundary detection capability—the BAA module forces the network to preserve sharp high-frequency edge gradients along water-film boundaries through directional Sobel-based feature pooling. Incorporating GMF alone achieves 95.42%, as the module dynamically recalibrates multi-scale features to prevent small puddled regions from being overwhelmed by global dry road contexts. With both modules integrated, the improved SegFormer attains the highest mIoU of 95.80%, accuracy of 97.84%, and F1 score of 97.86%, though FPS decreases slightly to 20.30—still meeting real-time requirements. These results validate that the synergistic coupling of GMF and BAA yields complementary benefits: GMF directs attention toward hazardous regions, while BAA refines boundary delineation.

4.2. Comparison with Classical Semantic Segmentation Models

To further validate the advantages of the proposed improved SegFormer model in highway slippery pavement identification, nine mainstream semantic segmentation models were selected for comparison, including Mask2Former, U-Net, DeepLabv3+, PSPNet, RTFNet, MFNet, and SNE-RoadSeg. All experiments were conducted on the same dataset under identical training epochs and optimization configurations to ensure fair-ness and comparability.

As presented in

Table 3. Comparisons between different semantic segmentation models.

Method	mIoU (%)	F1 Score (%)	Precision (%)	Recall (%)	AmIoU (%)
Mask2Former	93.84	96.82	97.14	96.51	74.80
U-Net	92.46	95.68	95.14	96.25	73.12
Deeplabv3+	93.82	96.81	96.99	96.63	62.66
PSPNet	94.06	96.33	96.12	96.54	70.10
RTFNet	93.10	96.90	96.30	97.60	49.60
MFNet	92.10	95.90	94.10	97.70	49.30
SNE-RoadSeg	93.80	96.80	96.10	97.50	53.40
Improved SegFormer	95.80	96.90	96.02	97.79	70.91

, the proposed improved SegFormer achieves the highest mI-oU of 95.80%, substantially outperforming PSPNet (94.06%), Mask2Former (93.84%), DeepLabv3+ (93.82%), and SNE-RoadSeg (93.80%)—corresponding to relative improve-ments of 1.74%, 1.96%, 1.98%, and 2.00%, respectively. While alternative networks like Mask2Former suffer from grid-like artifacts or over-smoothing along water-film bounda-ries, our BAA module preserves sharp high-frequency edge gradients, and the GMF mod-ule prevents small puddled regions from being overwhelmed by global dry road contexts. In terms of F1 score, the proposed model ties with RTFNet at 96.90%. Regarding AmIoU, which reflects cross-scenario generalization, the improved SegFormer attains 70.91%, ranking second only to Mask2Former (74.80%) and exceeding PSPNet (70.10%) and DeepLabv3+ (62.66%). In contrast, U-Net yields notably lower mIoU (92.46%) and F1 score (95.68%), with performance degradation under complex scenarios. Overall, the proposed method demonstrates a favorable balance across all metrics, confirming its superiority for highway slippery-pavement recognition tasks.

From the overall results, the improved SegFormer-based method achieves leading performance in both mIoU and other key metrics, while maintaining stable pixel-level ac-curacy. It particularly excels in the identification of small-area puddled regions, demon-strating significantly higher accuracy compared to other models. Leveraging its global modeling capabilities and multi-scale fusion mechanism, the proposed SegFormer more effectively distinguishes slippery regions from non-target backgrounds. Furthermore, de-spite adopting a Transformer-based architecture, the decoder of SegFormer remains highly streamlined, ensuring that the model maintains high inference efficiency with an average frame rate approaching 71 FPS, thereby meeting the real-time requirements of practical traffic monitoring systems.

To provide a comprehensive evaluation of the proposed framework, we further com-pare its computational complexity with several representative semantic segmentation models in terms of the number of parameters (Params) and floating point operations (FLOPs), as summarized in

Table 4. Comparison of parameters and FLOPs for different segmentation models.

Method	Params (M)	FLOPs (G)
Mask2Former	56.7	78.5
U-Net	38.8	62.6
Deeplabv3+	41.5	56.0
PSPNet	64.2	148.3
RTFNet	47.1	75.9
MFNet	68.6	127.8
SNE-RoadSeg	71.9	108.1
Improved SegFormer	60.5	116.8

Notes: The reported Params and FLOPs are collected from the original publications or official implementations under their respective experimental settings and are provided for reference only.

. The proposed improved SegFormer contains 60.5 M parameters and requires 116.8 G FLOPs. Compared with lightweight architectures such as U-Net and DeepLabv3+, the proposed model incurs a higher computational cost due to the introduction of the GMF and BAA modules. However, its complexity remains compa-rable to that of other advanced segmentation networks, demonstrating a favorable trade-off between computational complexity and segmentation performance. Considering that the target application is highway-pavement monitoring, where inference is typically performed on edge-computing platforms or roadside servers with moderate computation-al resources, the computational cost is considered acceptable for practical deployment.

4.3. Results on Different Datasets

In order to comprehensively evaluate the semantic perception performance of the proposed improved SegFormer-based method under diverse road scenarios, further vali-dations were conducted on various representative public datasets in addition to the da-taset obtained from the Guangzhou Beierhuan Expressway. The datasets used in this section mainly include the SYN-UDTIRI and Cityscapes datasets.

Training and validation were mainly conducted on the SYN-UDTIRI dataset, which is a high-quality dataset synthetically designed for road defect detection. The SYN-UDTIRI dataset provides RGB images paired with surface normal maps and rich pixel-level se-mantic annotations, covering various typical road conditions such as water accumulation, cracks, potholes, obstacles, and fog. Besides the synthetic SYN-UDTIRI dataset, transfer learning experiments were also conducted on real-world datasets such as Cityscapes to validate the model’s generalization capabilities. Cityscapes includes 5000 high-resolution street scene images with pixel-level annotation across 19 urban traffic-related categories.

Performance metrics of different semantic segmentation models on the SYN-UDTIRI dataset are presented in

Table 5. Comparative performance of improved SegFormer against classical methods [34] on the SYN-UDTIRI dataset.

Subset	Method	mIoU (%)	F1 Score (%)	Precision (%)	Recall (%)
Validation Set	Mask2Former	64.29	78.27	83.0	74.05
	SegFormer	52.46	68.82	70.13	67.55
	Deeplabv3+	52.94	69.23	75.23	64.12
	HRNet	52.92	69.21	79.46	61.30
Test Set	Mask2Former	46.91	63.87	73.59	56.41
	SegFormer	36.34	53.31	57.23	49.89
	Deeplabv3+	34.76	51.58	62.54	43.90
	HRNet	35.47	52.37	69.09	42.16
	SNE-RoadSeg	92.00	95.80	96.30	95.40
	RTFNet	90.30	94.90	94.10	95.70
	MFNet	89.50	94.50	95.70	93.30
	Improved SegFormer	93.35	96.56	96.53	96.59
	SNE-RoadSeg	92.10	95.90	96.70	95.10
	RTFNet	90.50	95.00	95.50	94.50
	MFNet	87.70	93.50	96.20	90.90
	Improved SegFormer	93.51	96.65	96.61	96.69

. As shown in Table 5, the improved SegFormer-based method proposed in this paper demonstrates significant advantages over many other models [34], achieving a Mean Intersection over Union (mIoU) of 93.51% and an F1 score of 96.65%.

As shown in

Table 6. Comparative performance of improved SegFormer against classical methods [34] on the Cityscapes Dataset.

Method	mIoU (%)	F1 Score (%)	Precision (%)	Recall (%)	AmIoU (%)
Mask2Former	93.84	96.82	97.14	96.51	74.80
SegFormer	93.98	96.90	96.02	97.79	64.51
Deeplabv3+	93.82	96.81	96.99	96.63	68.66
HRNet	94.06	96.94	96.29	97.59	70.10
SNE-RoadSeg	93.80	96.80	96.10	97.50	53.40
RTFNet	94.10	96.90	96.30	97.60	49.60
MFNet	92.10	95.90	94.10	97.70	49.30
Improved SegFormer	95.80	97.86	97.74	97.97	76.20

, the performance metrics of the proposed improved SegFormer-based method are greater than those of other methods [34], indicating that the proposed model achieves stronger comprehensive performance on the Cityscapes dataset. Specifically, the mIoU, F1 score, and AmIoU of the proposed improved SegFormer-based method are 95.80%, 97.86%, and 76.20%, respectively. The AmIoU of the proposed method signifi-cantly surpasses that of other methods. Notably, while HRNet yields higher recall due to its continuous high-resolution stream, our method achieves superior balance with signif-icantly higher precision, as the BAA and GMF modules effectively suppress false positives from specular reflections and shadows. In highway engineering applications, mitigating such false alarms is critical to prevent erratic automated speed-limit warnings. Experi-mental results on the Cityscapes dataset show that the proposed method possesses stronger generalization capabilities in complex scenarios.

4.4. Visualization Analysis

In this section, some representative images are selected to conduct visualization comparisons, in order to further illustrate the semantic segmentation capabilities of the proposed improved SegFormer-based method. Typical results are presented in Figure 6. As shown in Figure 6, the proposed method provides very precise segmentation results on typical water accumulation scenarios, demonstrating strong pixel-level spatial perception. The proposed model accurately delineates the splash traces and water film boundaries caused by vehicles, with segmentation contours closely aligning with the true water trace edges—even under low-contrast or reflective conditions. Moreover, the proposed method reliably distinguishes between slippery and dry regions with minimal mis-segmentation under light wet conditions, indicating robust performance and discriminative capability in handling low-contrast details. When confronted with complex background interferences such as rain-induced reflections, ground shadows, or vehicle occlusions, the improved SegFormer maintains stable segmentation performance, effectively suppressing background noise and enhancing prediction consistency and boundary precision compared to other classical models.

Heatmaps are introduced as an auxiliary visualization tool to express salient regions and perform graded analysis of the segmentation results, as illustrated in Figure 6. Heatmaps provide an intuitive representation of the model’s attention intensity on slippery regions and enable preliminary quantification of varying slipperiness levels. In puddled areas, the heatmap exhibits high-intensity responses, reflecting the model’s heightened sensitivity to hazardous regions. In dry or slightly wet areas, the response values gradually diminish, demonstrating strong discriminative capability. Although the current heatmap approach still has room for improvement in precision grading, it already exhibits a certain level of slipperiness perception, providing a scalable foundation for subsequent slipperiness level classification and early warning mechanisms.

As shown in Figure 7, on the real-world Cityscapes dataset [34], the improved SegFormer-based method also demonstrates excellent transfer performance. Without using additional pre-trained weights, the model achieves near State-of-the-Art results on target categories such as roads, buildings, and vehicles. It maintains stable outputs even in street scene images with blurred boundaries and severe occlusions, highlighting the strong generalization and adaptability of the proposed fusion mechanism. This advantage primarily stems from the model’s effective perception of normal structures during the encoding stage, enabling more accurate differentiation between drivable and non-drivable area boundaries.

Although the proposed framework demonstrates strong segmentation performance under most highway scenarios, several challenging conditions may still lead to performance degradation. Representative failure cases are presented in Figure 8. These include nighttime scenes with insufficient illumination, headlight glare, strong specular reflections on wet pavements, and severe shadow occlusions. Under these conditions, the visual contrast between slippery and non-slippery regions is significantly reduced, making boundary localization more difficult and occasionally resulting in incomplete or inaccurate segmentation. In addition, the model may exhibit reduced generalization performance on road surfaces with visual characteristics that differ substantially from those represented in the training dataset, such as snow-covered pavements, icy roads, or uncommon pavement materials. Future work will focus on expanding the diversity of the training dataset, incorporating domain adaptation techniques, and exploring multimodal sensing approaches to improve robustness under challenging environmental conditions.

5. Discussion

From a qualitative perspective, the superior performance of the proposed method stems from the synergistic coupling of GMF and BAA. While alternative deep networks like Mask2Former suffer from grid-like artifacts or over-smoothing along water-film boundaries, our BAA module forces the network to preserve sharp high-frequency edge gradients. Meanwhile, the GMF module dynamically recalibrates multi-scale features, preventing small puddled regions from being overwhelmed by global dry road contexts. This study proposes an improved SegFormer-based framework for slippery pavement recognition, integrating a Boundary-Aware Attention (BAA) module for precise edge delineation and a Guided Multi-scale Feature Fusion (GMF) module incorporating contextual information such as vehicle dynamics and splash-zone distribution. As demonstrated in Figure 6, the BAA module enables accurate segmentation of splash traces and water film boundaries even under low-contrast or reflective conditions, with contours closely aligning with ground truth edges. The GMF module directs attention toward hazardous regions, as evidenced by high-intensity heatmap responses in puddled areas. Together, these modules achieve robust segmentation of dry, wet, and puddled surfaces under uneven illumination, water-film reflections, and vehicle occlusions. As shown in Figure 7, the improved SegFormer-based method demonstrates strong transferability on the Cityscapes dataset without additional pre-trained weights, maintaining stable performance under blurred boundaries and severe occlusions. Comparative experiments (Table 5 and Table 6) confirm its superiority in mIoU, F1 score, and pixel-level accuracy, validating its effectiveness, accuracy, and stability across diverse scenarios.

Despite the high accuracy achieved by the proposed framework, domain shift remains a significant challenge for practical deployment. Variations in camera height, viewing angle, pavement materials, illumination, weather conditions, camera viewpoints, and image-acquisition devices can substantially alter the visual characteristics of slippery regions, thereby affecting segmentation performance. In particular, differences in pavement types, such as concrete and open-graded friction courses, as well as adverse winter conditions involving snow and ice, may introduce visual patterns underrepresented in the current dataset. Furthermore, nighttime scenarios with low illumination or headlight glare can weaken boundary and texture information, reducing segmentation accuracy. The proposed GMF and BAA modules improve feature discrimination and boundary representation, which enhances robustness under moderate domain variations. Nevertheless, their effectiveness may decrease in substantially different environments, such as snow-covered roads or severe nighttime glare, where visual appearance differs significantly from the training distribution. Future work will focus on incorporating unsupervised domain adaptation techniques and expanding the training dataset to include a wider range of road, weather, and lighting conditions, thereby improving robustness and cross-domain generalization. Additionally, further validation will be conducted across a larger number of geographic locations and varied climatic conditions, with a specific focus on cross-referencing visual recognition results with physical slipperiness indicators such as British Pendulum Numbers (BPN) or continuous friction coefficients.

6. Conclusions

This paper proposes an improved SegFormer-based method for highway slippery pavement recognition by introducing a Boundary-Aware Attention (BAA) module and a vehicle-semantic-guided Guided Multi-scale Fusion (GMF) module, together with a multi-scale input strategy and a compound loss function. Main conclusions are summarized as follows:

(1)

The integration of BAA and GMF modules significantly boosts model performance, with BAA refining edge delineation of water films and splash traces and GMF leveraging vehicle semantic cues (e.g., vehicle type, speed level, splash locations) to enhance focus on critical regions, improving mIoU from 94.11% (baseline) to 95.80% and achieving an F1 score of 97.86%.

(2)

On the self-constructed highway dataset and the Cityscapes benchmark, the proposed method achieves 95.80% and 95.80% mIoU, respectively, outperforming mainstream models such as Mask2Former and PSPNet and demonstrating strong cross-scene generalization under diverse real-world conditions.

(3)

The method maintains a high inference speed of 70.91 FPS while delivering accurate segmentation of dry, wet, and puddled surfaces, satisfying real-time requirements and offering a practical solution for highway driving safety management.

Despite these promising results, domain shift remains a key challenge for practical deployment, and future work will focus on domain-adaptation techniques, dataset expansion, and cross-referencing with physical friction measurements, as discussed in detail in Section 5.