PPDD: Egocentric Crack Segmentation in the Port Pavement with Deep Learning-Based Methods

Abstract

Road infrastructure is a critical component of modern society, with its maintenance directly influencing traffic safety and logistical efficiency. In this context, automated crack detection technology plays a vital role in reducing maintenance costs and enhancing operational efficiency. However, previous studies are limited by the fact that they provide only bounding box or segmentation mask annotations for a restricted number of crack classes and use a relatively small size of datasets. To address these limitations and advance deep learning-based crack segmentation, this study introduces a novel crack segmentation dataset that reflects real-world road conditions. The proposed dataset includes various types of cracks and defects—such as slippage, rutting, and construction-related cracks—and provides polygon-based segmentation masks captured from an egocentric, vehicle-mounted perspective. Using this dataset, we evaluated the performance of semantic and instance segmentation models. Notably, SegFormer achieved the highest Pixel Accuracy (PA) and mean Intersection over Union (mIoU) for semantic segmentation, while YOLOv7 exhibited outstanding detection performance for alligator crack class, recording an ${AP}_{50}$ of 87.2% and $AP$ of 57.5%. In contrast, all models struggled with the reflection crack type, indicating the inherent segmentation challenges. Overall, this study provides a practical and robust foundation for future research in automated road crack segmentation. Additional resources including the dataset and annotation details can be found at our GitHub repository.

1. Introduction

Roads are among the most essential infrastructure for maintaining the economic and physical connectivity of society, and their quality and stability have both direct and indirect impacts on economic and social issues [1]. Pavement surfaces gradually deteriorate due to factors such as climate conditions, traffic volume, and vehicle loads, which can negatively affect traffic safety and the efficient operation of road networks [2]. Since pavement maintenance and repair strategies are typically tailored to the causes of cracking, accurate identification of crack types is a critical task. Fatigue cracking in pavements arises from diverse failure mechanisms and exhibits distinctive visual characteristics that reflect the underlying causes. Research on such pavement surface defects enables appropriate maintenance actions and supports regular monitoring and interventions to preserve road quality and stability. These efforts, in turn, bring about positive societal outcomes such as improved traffic safety, reduced vehicle maintenance costs, and enhanced pedestrian convenience [3].

Traditional pavement inspection methods have mainly relied on visual assessments by human inspectors or semi-automated techniques, where road surface measurements are collected using automated equipment and later reviewed manually. These methods are inherently limited by their reliance on subjective judgment [4] and often involve considerable time and financial costs. In addition, inspectors must occupy and control traffic lanes during inspections, increasing the risk of accidents and traffic disruptions. To overcome these limitations, deep learning-based detection models have gained traction in recent years, significantly advancing automated crack detection technologies [1]. However, training such models typically requires densely annotated crack labels, which poses a major barrier to large-scale adoption. Consequently, existing studies have mostly relied on bounding box annotations for entire images [5] or segmentation masks applied at the patch level (e.g., cropped images of cracks) [1,6].

In this study, we propose a novel benchmark dataset for deep learning-based crack detection. The dataset includes previously under-represented crack types such as slippage, rutting, and construction but also provides polygon-based segmentation masks for enhanced annotation precision. Unlike existing crack segmentation datasets, which often rely on orthographic imaging—capturing cracks from a perpendicular viewpoint on surfaces such as bridges or building exteriors—our dataset adopts an egocentric acquisition method, capturing road cracks from the viewpoint of a moving vehicle. This acquisition setup allows the dataset to include oblique views and real-world perspective distortions, thereby more accurately reflecting the visual conditions encountered in practical, on-road applications. In particular, we constructed a new benchmark segmentation dataset for road cracks and expanded the crack taxonomy to encompass a wider variety of crack types. Furthermore, by conducting a comprehensive evaluation of various segmentation architectures using the proposed dataset, we demonstrated both its practical utility and its potential as a standardized benchmark for future research.

To establish baselines and analyze the characteristics of the proposed road crack segmentation dataset, we evaluate the performance of deep learning-based detection models on both semantic and instance segmentation tasks. For semantic segmentation, we adopt three representative models: Mask2Former [7], SegFormer [8], and Deeplabv3+ [9]. For instance segmentation, we utilize YOLOv7 [10] and Mask2Former [7]. Our experimental results provide both quantitative and qualitative evaluations of the proposed dataset, demonstrating its potential for training and benchmarking various baseline models.

Overall, the proposed dataset offers a diverse set of crack categories along with detailed segmentation annotations. Our benchmark evaluations on both semantic and instance segmentation models not only establish baseline performance but also highlight future research directions. Section 2 reviews existing road defect datasets and related deep learning models. Section 3 details the acquisition and preprocessing of our dataset. In Section 4, we present our experimental results, and in Section 4 and Section 5, we discuss future directions and conclude this paper.

2. Related Work

2.1. Public Dataset for Crack Identification and Detection

Public datasets for analyzing the visual characteristics of cracks using deep learning-based visual perception models can largely be categorized into two research directions. The first approach focuses on segmentation for detailed representation of cracks. This involves identifying fine-grained visual features of cracks that appear in diverse contexts—such as roads, bridges, and concrete building facades—under varying causes of deterioration. To achieve this, crack segmentation datasets typically involve orthographic imaging, in which the surface affected by cracking is captured from a perpendicular viewpoint. From these images, crack regions are cropped into patches, and segmentation masks are annotated to delineate the fine fissures present in the crack areas [11,12]. While this approach enables detailed analysis of the visual characteristics and precise delineation of various types of cracks, it has limitations when applied to the development of automated road crack detection systems. The second research direction focuses on crack detection specifically tailored for road surfaces. This approach utilizes annotations of road cracks captured from an ego-vehicle view, providing data that more closely resemble real-world driving scenarios. Recently, the SHREC 2022 Challenge [1] addressed this topic by organizing a task focused on detecting potholes and comprehensive cracks in road pavement imagery. The participants of the competition analyzed the challenges of road surface defect detection and proposed solutions, thereby contributing to the advancement research on deep learning-based detection systems using road-specific datasets. However, existing public datasets for road crack detection are limited in terms of both the diversity of crack categories and the availability of pixel-level dense annotations. Most of these datasets only provide bounding box annotations indicating the location of cracks within ego-vehicle images [13,14]. EdmCrack600 [15,16] attempted to address the limitations of front-mounted ego-vehicle views—such as obstructions caused by the vehicle hood—by mounting a GoPro camera on the rear of the vehicle. While the dataset includes segmentation annotations and real-world road surface data, its scale is limited to only 600 images. In

Table 1. Comparison of road crack datasets.

Data	Image Type	Task	Number of Class	Dataset Size
Chen, T., et al. [17]	Patch	SEG	1	10,000 (5000/2500/2500)
EdmCrack600 [16]	Ego-vehicle	SEG	1	600 (420/60/120)
CPRID	Ego-vehicle	SEG	1	2235 (2000/200/35)
RCDC [18]	Ego-vehicle	OD	4	23,705 (18,930/2111/2664)
RDD [19]	Ego-vehicle	OD	8	9053 (-)
Omicrack30k [6]	Patch	SEG	1	30,017 (22,158/3277/4582)
PPDD (Ours)	Ego-vehicle	SEG	6	204,839 (163,871/20,484/20,484)

Image type: perspective of the captured images (e.g., patch-based, ego-vehicle view); task: type of annotation task (SEG: semantic segmentation, OD: object detection); dataset size: number of images (training/validation/test splits).

, we compare our dataset with existing road damage datasets in terms of data types, number of classes, types of deep learning tasks applied, and dataset size. This comparison reveals that, unlike previous datasets, which often focus on specific damage types or limited tasks, the dataset constructed in this study encompasses a wider range of data types and a greater number of classes. It also demonstrates scalability and flexibility for application to various deep learning tasks, including classification and detection. To enable more practical and challenging road crack analysis, we propose a new benchmark dataset consisting of ego-vehicle images annotated with segmentation masks for six distinct crack categories.

2.2. Deep Learning-Based Segmentation

Dense prediction in deep learning-based object recognition systems can be broadly categorized into object detection tasks, which identify the spatial location of target objects [20], and object segmentation tasks [21], which further predict pixel-level masks for the detected objects. In particular, image segmentation tasks are divided into semantic segmentation [21], which performs pixel-level classification by predicting the presence of object categories in both foreground and background and instance segmentation [22], which predicts separate segmentation masks for each individual object instance. A typical semantic segmentation process consists of an encoder that extracts pixel-level visual representations and a decoder that reconstructs the prediction masks to match the original image resolution. SegFormer [8], in particular, introduces a hierarchical transformer-based encoder designed to capture both coarse and fine features, along with a lightweight All-MLP decoder that fuses multi-level features for accurate mask prediction. DeepLabv3+ [9] utilizes Atrous Spatial Pyramid Pooling (ASPP) to effectively capture objects at multiple scales and introduces a decoder module that enhances the reconstruction of fine object boundaries. Instance segmentation tasks are typically implemented by attaching a mask-specific decoder to the instance representations predicted by a detection agent. YOLOv7 [10] is an advanced version of the original one-stage detection pipeline, YOLO [20], with multiple architectural improvements. By incorporating a dedicated segmentation mask decoder, it enables segmentation prediction while preserving the fast inference speed characteristic of YOLO-based models. Mask2Former [7] proposes a unified architecture for any image segmentation task—including semantic, instance, and panoptic segmentation—by introducing masked attention for cross-attention between predicted mask regions and a transformer decoder that efficiently leverages high-resolution features from a pixel decoder.

Recent advances in transformer-based segmentation models and domain-specific detectors have led to the development of increasingly powerful dense prediction techniques. For instance, OneFormer [23] introduces a multi-task unified transformer architecture that utilizes task-specific conditional queries, achieving superior performance across semantic, instance, and panoptic segmentation tasks compared to individually trained models such as Mask2Former. In the field of pavement crack analysis, Guo et al. [24] proposed the “Crack Transformer”, which leverages a Swin Transformer encoder to significantly improve pixel-level crack segmentation under challenging conditions such as shadows and complex crack structures, outperforming traditional CNN-based methods like DeepLabv3+. These recent studies highlight the continuous progress in dense prediction—from general-purpose segmentation transformers to task-specific crack detection networks—and underscore the importance of scalable and representative datasets to support further advancement.

3. Materials and Methods

3.1. Port Pavement Distress Dataset (PPDD)

The data were collected from roadways near the port area of Busan, South Korea. This region is characterized by high traffic volumes, including frequent passage of heavy-freight vehicles, which increases the likelihood of various types of cracking and road surface damage [25]. In this study, we systematically observed and collected a wide range of pavement defects occurring in such asphalt road environments.

The dataset used in this study was constructed from video data captured during driving using a GoPro10 camera (CMOS sensor, 23.0MP, 5.3K resolution at 60 fps; GoPro, Inc., San Mateo, CA, USA) mounted on the upper front section of a vehicle. Images were recorded at a resolution of 3840 × 2160 pixels, and only frames containing at least one visible road defect were selected for analysis. The recordings were conducted during three time periods (10:00–12:00, 13:00–15:00, and 15:00–17:00) across different days and under two weather conditions: clear and overcast. As illustrated in Figure 1, crack and defect data were extracted from the lower part of each frame—specifically, a region extending 1000 pixels upward from the bottom edge of the image. This area corresponds to the rear view of the vehicle and includes the left, center, and right lanes of the roadway. The red polygon highlighted in the figure indicates the region used for annotation (labeling); however, this area was not cropped for use during model training or evaluation. To ensure privacy protection, personally identifiable elements such as vehicle license plates and commercial signage were anonymized in accordance with data protection standards.

The dataset constructed in this study was designed to comprehensively cover various types of cracks and defects commonly found on port roads. A total of six major crack categories were defined and labeled: reflective cracking (RC), longitudinal and edge cracking (LEC), corrugation, shoving, and bleeding cracking (CSSC), rutting and depression cracking (RDC), construction joint cracking (CJC), and alligator cracking (AC). In total, 204,839 images were collected and used for analysis. The distribution of labeled instances across the dataset is as follows: 124,780 instances of RC, 183,094 of LEC, 20,210 of CSSC, 68,434 of RDC, 109,625 of CJC, and 88,571 of AC, resulting in a total of 594,714 labeled crack instances. Representative visual examples of each crack category are provided in Figure 2.

In this study, we aim to detect and analyze surface defects and cracks on roadways using semantic and instance segmentation techniques. To this end, the dataset was annotated using polygon-based labeling and crack region masks, following the annotation format of the MS COCO dataset [26]. For semantic segmentation, the polygon annotations for each crack instance were converted into binary masks distinguishing background and object regions. Every pixel within the annotated regions was assigned a class label corresponding to one of the six crack categories. The labeling process was assisted by the use of the Computer Vision Annotation Tool (CVAT) [27], an automated annotation platform. The automatically generated labels were manually verified on a frame-by-frame basis to ensure accuracy. While inter-annotator agreement was not formally measured during dataset construction, we plan to evaluate annotation consistency using a subset of samples in a follow-up study. Figure 3 presents sample images illustrating the mask annotations for semantic segmentation and polygon annotations for instance segmentation. The constructed dataset reflects the distribution of crack types found on port-area roads and provides a reliable and generalizable foundation for real-world crack detection and analysis tasks.

3.2. Baselines

Using the Port Pavement Distress Dataset (PPDD), we adopted Mask2Former, SegFormer, and DeepLabv3+ for semantic segmentation, as these models have demonstrated strong performance in fine-grained object segmentation tasks. For instance segmentation, we employed YOLOv7 and Mask2Former, chosen for their flexible architectures capable of performing both detection and segmentation simultaneously with high accuracy. All experiments were conducted at a fixed input resolution of 640 × 640. This resolution was selected based on two considerations: (1) it matches the input size used by widely adopted real-time detection models such as YOLOv5 and YOLOv7 [10], and (2) it offers a balanced trade-off between computational efficiency and segmentation performance. We therefore chose this resolution to ensure comparability with standard practices while maintaining feasibility for future deployment in resource-constrained systems. Model training and visualization were conducted using the MMDetection [28] and MMSegmentation [29] frameworks within the OpenMMLab ecosystem. Model performance was evaluated using standard metrics: Pixel Accuracy (PA) and mean Intersection over Union (mIoU) for semantic segmentation and mean Average Precision (AP) for instance segmentation. In addition, qualitative comparisons were conducted to assess the precision of crack boundary segmentation and to examine model-specific differences.

4. Experiments

4.1. Experimental Setup

In this study, the road crack dataset was split into training, validation, and test sets in a ratio of 8:1:1 to enable effective model training and evaluation. The test set was only used exclusively to assess the generalization performance of the segmentation models. Considering GPU memory constraints and training stability, the batch size was set to 64 and the learning rate to 0.0001. Training each epoch took approximately 60 min, although this may vary depending on GPU performance, dataset size, and model complexity. Model training was performed using two NVIDIA Ada 6000 GPUs in parallel, on a system running Ubuntu 20.04 with an Intel Xeon Silver CPU and 256 GB of RAM. The code and experimental settings used for training and evaluation are publicly available on GitHub as follows: https://github.com/LeaYoon/PPDD (accessed on 1 April 2025).

4.2. Experimental Results

4.2.1. Quantitative Evaluation

To compare semantic segmentation performance across different types of pavement cracks, we conducted an evaluation using three models with distinct feature extraction mechanisms: Mask2Former, SegFormer, and DeepLabv3+. To compare semantic segmentation performance across different types of pavement cracks, we conducted an evaluation using three models with distinct feature extraction mechanisms: Mask2Former, SegFormer, and DeepLabv3+. For Mask2Former, we experimented with two backbone networks: ResNet-101 and Swin Transformer. SegFormer uses MiT-B2, and DeepLabv3+ is implemented with ResNet-50.

Table 2. Quantitative evaluation results for each class and overall performance in semantic segmentation.

	(%)	RC	LEC	CSSC	RD	CJC	AC	Background	Total
Mask2Former-r101
	PA	48.71	53.9	42.5	69.33	60.33	64.39	97.46	62.37
	mIoU	33.8	38.7	29.13	36.79	40.31	54.62	95.42	46.96
Mask2Former-swin-l
	PA	48.45	56.03	40.67	64.75	62.9	65.84	97.98	62.37
	mIoU	34.36	41.82	34.92	39.89	41.76	56.19	95.90	49.26
SegFormer
	PA	40.84	58.64	53.92	73.10	58.17	81.96	99.10	66.53
	mIoU	35.13	50.74	48.31	62.27	49.19	72.55	97.52	59.39
DeepLabv3+
	PA	39.76	48.67	36.29	46.54	46.45	68.38	98.70	54.97
	mIoU	31.95	38.14	27.05	37.86	36.93	57.18	96.19	46.47

Performance metrics are expressed as percentages and rounded to two decimal places, RC: reflection cracking, LEC: longitudinal–edge cracking, CSSC: corrugation, shoving, and slippage cracking, RD: rutting and depression, CJC: construction joint cracking, AC: alligator cracking.

presents the evaluation results on the test set, reporting Pixel Accuracy (PA) and Intersection over Union (IoU) scores across six crack categories for all four model configurations. In this study, we employed two standard metrics—Pixel Accuracy (PA) and mean Intersection over Union (mIoU)—to evaluate the performance of semantic segmentation models. Pixel Accuracy measures the ratio of correctly classified pixels to the total number of pixels in the image, providing an overall sense of classification accuracy. In contrast, mIoU quantifies the average overlap between the predicted and ground truth segmentation masks. It is calculated by computing the Intersection over Union (IoU) for each class, defined as the intersection area divided by the union area of predicted and true regions, and then averaging across all classes. mIoU is particularly regarded as a more robust metric in scenarios with class imbalance, as it accounts for the quality of segmentation on a per-class basis. Among the models, SegFormer demonstrated the highest semantic segmentation performance in terms of both PA and mIoU, achieving particularly high accuracy (97.52%) on the background class. In per-class performance, the CSSC category showed the lowest prediction scores in three out of the four models, while the AC type exhibited relatively higher accuracy. Overall, the transformer-based models—SegFormer and Mask2Former with a Swin Transformer backbone—achieved superior performance in semantic segmentation. These results highlight the effectiveness of transformer-based architectures in producing fine-grained masks for pavement crack and defect detection.

To evaluate instance segmentation performance across different types of pavement cracks, we compared the Mask2Former and YOLOv7 models. For Mask2Former, experiments were conducted using two different backbone networks: ResNet-101 [30] and Swin Transformer [31].

Table 3. Quantitative evaluation results for each class and overall performance in instance segmentation.

	(%)	RC	LEC	CSSC	RD	CJC	AC	Total
Mask2Former-r101
	${\mathrm{A}\mathrm{P}}_{50}$	45.9	50.7	49.1	62.6	64.9	69.7	57.15
	$\mathrm{A}\mathrm{P}$	15.4	17.1	18.7	36.6	29.6	43.3	26.78
Mask2Former-swin-s
	${\mathrm{A}\mathrm{P}}_{50}$	51.9	56.5	57.6	66.7	70.10	76.3	63.18
	$\mathrm{A}\mathrm{P}$	18.5	20	22.7	41.4	34.6	49.4	31.10
YOLOv7
	${\mathrm{A}\mathrm{P}}_{50}$	57.4	65.1	80.2	84	84.3	87.2	76.37
	$\mathrm{A}\mathrm{P}$	21.3	25.2	38.3	55.9	45.2	57.5	40.57

Performance metrics are expressed as percentages and rounded to two decimal places, RC: reflection cracking, LEC: longitudinal–edge cracking, CSSC: corrugation, shoving, and slippage cracking, RD: rutting and depression, CJC: construction joint cracking, AC: alligator cracking.

presents the ${\mathrm{A}\mathrm{P}}_{50}$ and overall $\mathrm{A}\mathrm{P}$ scores for the three model configurations across the six crack categories in the test dataset. ${\mathrm{A}\mathrm{P}}_{50}$ (Average Precision at an IoU threshold 50%) measures the accuracy of predictions with an Intersection over Union greater than 0.5. In contrast, $\mathrm{A}\mathrm{P}$ refers to the mean Average Precision calculated across IoU thresholds from 0.5 to 0.95 in increments of 0.05, following the COCO evaluation metric [32], providing a more comprehensive assessment of overall model performance. Among the models, YOLOv7 achieved the best overall performance, demonstrating particularly strong results on the AC category, with an ${\mathrm{A}\mathrm{P}}_{50}$ of 87.2% and an $\mathrm{A}\mathrm{P}$ of 57.5%. On the other hand, the RC category yielded the lowest performance across all three models, suggesting that this type of crack is comparatively more challenging to segment and detect than others. Although the CSSC class accounts for only 3.3% of the dataset, its segmentation performance is not the lowest among the six crack types. For instance, the YOLOv7 model achieves an AP of 38.3% for CSSC, which is notably higher than that of the RC class (21.3%), despite the RC class occupying a larger portion of the dataset. This suggests that the challenge of segmentation may depend not only on class frequency but also on visual distinctiveness and contextual cues.

4.2.2. Qualitative Evaluation

Figure 4 presents a visual comparison of how accurately the semantic segmentation models distinguish different crack types in the PPDD dataset. Panel (a) shows the ground truth labels, depicting four types of pavement damage—RC, LEC, CJC, and AC—in their annotated forms. For RC, a horizontally oriented crack, SegFormer was the most effective in capturing the number and type of cracks, although it struggled with fine boundary delineation, as shown in (c). Mask2Former (d) also detected some portions of RC, but there were instances of omission and misclassification, particularly where RC was confused with AC. In the case of CJC, DeepLabv3+ (b) failed to deliver precise predictions. However, SegFormer (c), Mask2Former-r101 (d), and Mask2Former-swin-l (e) showed relatively detailed boundary representation and yielded outputs closely resembling the ground truth.

Figure 5 provides a visual comparison of instance-level segmentation performance, showing the predictions of YOLOv7 (b), Mask2Former with a ResNet-101 backbone (c), and Mask2Former with a Swin-L backbone (d), alongside the ground truth annotations (a). YOLOv7 demonstrated stable detection of AC, which spanned a wide area, and yielded relatively high confidence scores for the predicted instances (AC: 87.0, 89.8). However, it tended to miss finer details in CJC, indicating a limitation in capturing subtle damage patterns. In the case of Mask2Former, regardless of the backbone architecture, the model tended to overpredict the number of crack instances compared to the ground truth annotations. This suggests the presence of false-positive errors in the Mask2Former predictions.

Figure 6 visualizes the results of both semantic segmentation and instance segmentation on the same road image. It can be observed that cracks belonging to the RDC class are frequently misclassified as AC when they exhibit similar visual patterns. In both semantic and instance segmentation results ((a)–(c) and (d)–(f)), regions annotated as RDC are often predicted as AC, suggesting that the models tend to confuse the two classes when structural similarities are present. The predictions from Mask2Former (middle columns (b) and (e)) show a tendency to over-detect AC, RC, and LEC classes across both tasks. Numerous additional crack segments—colored as AC, RC, or LEC—appear even in areas without corresponding ground truth annotations. This over-detection may be attributed to the model’s high sensitivity and the class imbalance in the training dataset. In contrast, YOLOv7 (Subfigure (f)) produces fewer predictions in regions without actual cracks, whereas Mask2Former frequently generates false positives in such areas. Mask2Former also demonstrates stable IoU scores and accurate class predictions for long, horizontally, or vertically stretched cracks, indicating its robustness in capturing linear structures and correctly classifying their types. YOLOv7, on the other hand, does not excessively overestimate the area of such elongated cracks but sometimes misinterprets their orientation. Since YOLOv7 outputs axis-aligned bounding boxes, vertical cracks may be enclosed within horizontally shaped boxes, and vice versa.

5. Discussion

5.1. Road Crack Detection and Segmentation Dataset

In selecting representative models for evaluation, we referred to established practices in prior studies on pavement crack analysis to ensure methodological consistency. Accordingly, we incorporated both CNN-based segmentation architectures (e.g., YOLO, DeepLab) and ViT-based segmentation architectures (e.g., Mask2Former, SegFormer), which are widely recognized and frequently adopted as core architectures in the recent literature. This selection enables meaningful benchmarking while maintaining relevance to previous research. Recent advancements in deep learning-based crack detection and segmentation technologies have been actively reported through competitive challenges [1], newly released datasets [6], and academic publications [33]. The fine-grained visual appearance of cracks—whether on natural surfaces or man-made structures—can vary significantly depending on the surrounding environment and the underlying causes of formation. OmniCrack30K [6] was developed to facilitate a comprehensive understanding of cracks by aggregating 30,000 diverse crack samples extracted from over 20 different datasets. The authors evaluated crack segmentation performance using both domain-specific models trained to represent crack-specific features and a group of general-purpose segmentation models, including DeepLabV3 [34] and Mask2Former [7]. Their findings revealed that the general-purpose models trained on the proposed dataset significantly outperformed the crack-specific models. This outcome highlights the critical importance of having access to a sufficiently large and well-curated dataset in advancing the performance of deep learning-based models. In the context of paved roads, cracks are not only defined by fissures but also by the overall visual degradation of the road surface, which constitutes an important visual characteristic for detection. This aspect is highly domain-specific and serves the independent purpose of supporting road maintenance through dedicated crack detection. To address this, the road damage dataset (RDD) [14] was introduced, providing annotated bounding boxes for classes such as longitudinal cracks, transverse cracks, alligator cracks, and potholes, collected from paved roads across multiple countries. This dataset has since supported the advancement of crack detection research through a series of ongoing competitions [5,18,35,36,37]. In the recent ORDDC2024 challenge [37], which utilized the RDD dataset, all of the top five teams adopted models from the YOLO series [20] as their backbone architectures, aiming to maximize detection performance relative to inference time. The highest-performing model achieved an overall F1-score of 79.27 across all participating countries. The SHREC2022 challenge [1] focused on evaluating semantic segmentation performance for various crack and pothole classes. The competition used Weighted Pixel Accuracy (WPA) and mean Intersection over Union (mIoU) as evaluation metrics. To facilitate performance comparison with submitted models, the organizers employed DeepLabv3+ [9] as a baseline. DeepLabv3+ achieved a WPA of 0.598 and an mIoU of 0.676, establishing a representative baseline and reference level of performance for benchmark studies in crack segmentation. In this study, we adopted widely used baseline models for crack segmentation—such as YOLOv7 [10], Mask2Former [7], and Deeplabv3+ [9]—which have been frequently employed in prior research for detection and semantic segmentation tasks. Based on our experimental results, we conclude that our work also establishes a valid and consistent baseline aligned with existing standards in the field.

5.2. Limitations and Opportunities

PPDD comes with several considerations for analysis. First, all data were collected by a single, consistent organization. In contrast, the RDD dataset [14] includes road crack images collected from multiple countries. This diversity introduces challenges in evaluating model robustness to non-visual cues or assessing few-shot learning performance, as it becomes difficult to isolate the effects of domain gaps related to country- or city-specific factors. Although this consistency ensures high-quality annotations, the lack of geographic variety in the PPDD dataset may limit its ability to represent road damage patterns observed in other regions or climates. In other words, PPDD alone may not be sufficient for evaluating a model’s generalization across geographic domains. However, it may serve as a valuable evaluation set for studies focusing on crack-centric visual characteristics or for investigating domain gap effects in a more controlled setting. Another important consideration is class imbalance. Since the categories and visual characteristics of road cracks are closely tied to their underlying causes, various environmental factors—such as the primary function of the road or the frequency of certain types of wear—can influence the distribution of classes within a dataset. The proposed dataset was collected from port roads in Busan, South Korea—a city with a high volume of container transport—making it possible to include crack categories associated with heavy loads, such as corrugation, shoving, and slippage as well as rutting and depression. However, the occurrence frequencies of these classes remain relatively low, comprising only 3.3% and 11.5% of the dataset, respectively. Given the active research within the deep learning community on mitigating learning bias caused by imbalanced data [38], this dataset can also serve as a benchmark for evaluating model performance in low-frequency class scenarios.

5.3. Future Directions

Our new benchmark dataset introduces several promising directions for future research in road crack detection and analysis. We organize the future directions into two broad categories: (1) domain-specific challenges within the expanded taxonomy of crack types, and (2) integration with emerging techniques in deep learning and computer vision. The high-quality segmentation annotations provided in PPDD offer detailed information on the extent of damage, enabling the development of more practical and real-world-applicable crack segmentation models. The dataset includes a diverse set of crack categories, contributing to the analysis of visual characteristics associated with different causes of pavement damage. First, it would be worthwhile to explore methods for enhancing model sensitivity to visually subtle cracks with specific directional characteristics, such as reflection cracks. These cracks frequently appear perpendicular to the driving lane and often exhibit low visual contrast with nearby road markings, making them prone to misclassification. To address this challenge, architectural improvements that explicitly encode directional information and effectively learn spatial context may serve as effective solutions. Proposed strategies include adapting transformer-based architectures to better capture global contextual features [39,40], as well as applying connected component-based post-processing techniques for prediction refinement [41]. These approaches aim to improve model robustness for complex crack types by leveraging both structural alignment and contextual dependencies inherent in road defect imagery. While prior work [42] has considered an mIoU of 0.5 to represent reasonably sufficient performance, our model achieves an mIoU of 48.31 for the rarest class in the dataset. Despite the extremely limited training opportunities for this class, the model is still able to capture meaningful patterns, indicating that it has learned to recognize the structural characteristics of even the most infrequent crack types. Future work applying a broader range of model architectures beyond the proposed baselines may further improve segmentation performance. Additionally, the use of ensemble techniques or hybrid architectures could enhance robustness across diverse crack types. Incorporating advanced data augmentation strategies and class balancing techniques [43] may also help mitigate issues arising from class imbalance and limited training samples, particularly for rare crack categories. Together, these approaches could lead to more accurate and generalizable models for real-world crack segmentation tasks. Moreover, although the model maintained reasonable performance under overcast conditions, the number of such samples was limited, and thus, a detailed condition-specific performance analysis was not included in this study. In future work, expanding annotations related to environmental factors such as weather and lighting, and introducing condition-aware evaluation protocols will be crucial to further improve model robustness under varying real-world scenarios. To support real-world applications, we also plan to expand the dataset with more comprehensive weather variation annotations. Secondly, recent advances in deep learning-based perception have increasingly focused on training predictive models under challenging constraints. This includes scenarios where only a limited number of accurate annotations are available, where annotations are weak [44,45] or incomplete [46,47] or where domain gaps [48] arise between training and deployment environments. The proposed crack segmentation dataset is well positioned to support emerging research directions driven by the expansion of deep learning technologies. One particularly promising avenue is the integration of vision–language models (VLMs), such as CLIP [49], which have been trained on large-scale image–text datasets. VLMs represent a significant advancement in the scope and applicability of AI perception, enabling models to leverage text-based alignment and commonsense reasoning in visual tasks. As studies increasingly explore the application of VLMs to perception tasks, their ability to transfer learned representations opens up new possibilities for identifying a broader range of crack categories. The diverse crack categories included in our dataset can thus serve as a critical benchmark for evaluating how well existing recognition models, including those based on VLMs, generalize to previously unseen or fine-grained categories. Although this study does not primarily aim for direct application in real-time or embedded systems, the experimental environment—conducted at a resolution of 640×640 (VGA level)—reflects a configuration similar to that commonly used in lightweight inference scenarios for embedded platforms. Therefore, while embedded deployment was not the explicit objective, the resolution-constrained evaluation offers practical implications relevant to such environments. Our focus was on assessing segmentation accuracy and crack-type differentiation performance across state-of-the-art models. In future work, we plan to explore the possibility of adapting the proposed methods to embedded and real-time applications through model compression, quantization, or other efficiency-oriented strategies.

Therefore, we expect that the dataset proposed in this study will serve as a valuable objective resource for informing road maintenance and repair strategies. Furthermore, it is anticipated to contribute to the evaluation of crack detection model performance and the assessment of their generalization capabilities in future research.

6. Conclusions

In this study, we constructed a high-precision crack segmentation dataset that reflects real-world road conditions from an ego-vehicle perspective. Based on this dataset, we evaluated the performance of both semantic and instance segmentation models. SegFormer and YOLOv7 demonstrated strong performance in semantic segmentation and in detecting specific crack categories, respectively. However, certain crack types—such as reflective cracking—remained difficult to detect accurately, highlighting ongoing challenges in fine-grained crack recognition. The proposed dataset contributes not only by providing detailed polygon-based annotations for a diverse range of crack categories, but also by offering data captured in realistic driving environments, which are under-represented in existing datasets. These features enhance the practical relevance of the benchmark and allow for more robust evaluation across models. We believe this work establishes a valuable foundation for future research in road surface defect analysis. In particular, future studies may build on this dataset to improve model generalization to unseen road textures, evaluate cross-domain transferability, and explore efficient annotation strategies to support large-scale training and benchmarking.