Improved YOLOv5 Lane Line Real Time Segmentation System Integrating Seg Head Network ^†

Abstract

With the rise in motor vehicles, driving safety is a major concern, and autonomous driving technology plays a key role in enhancing safety. Vision-based lane departure warning systems are essential for accurate navigation, focusing on lane line detection. This paper reviews the development of such systems and highlights the limitations of traditional image processing. To improve lane line detection, a dataset from Roboflow Universe will be used, incorporating techniques like priority pixels, least squares fitting for positioning, and a Kalman filter for tracking. YOLOv5 will be enhanced with a di-versified branch block (DBB) for better multi-scale feature extraction and an improved segmentation head inspired by YOLACT (You Only Look At CoefficienTs) for precise lane line segmentation. A multi-scale feature fusion mechanism with self-attention will be introduced to improve robustness. Experiments will demonstrate that the improved YOLOv5 outperforms other models in accuracy, recall, and mAP@0.5. Future work will focus on optimizing the model structure and enhancing the fusion mechanism for better performance.

1. Introduction

Rapid urbanization and industrialization have made small cars a popular means of transportation, playing a significant role in highway development and improving the living environment. However, the increasing number of vehicles worldwide has raised concerns about traffic safety. Technological advancements have improved life quality but have also led to issues such as traffic accidents, congestion, environmental pollution, and energy waste. Traffic accidents result in significant casualties and economic losses, with China alone experiencing over 200,000 deaths annually due to such incidents. To address these concerns, intelligent transportation systems (ITS) have been developed, incorporating people, vehicles, roads, and the environment to improve traffic safety and reduce accidents. While countries like the United States, Japan, and European nations have made strides in ITS and autonomous driving technologies, China’s automotive intelligence research is still in the early stages, focusing on assisted driving. Real-time lane segmentation, a critical component of assisted driving, can reduce accidents caused by driver inattention. However, traditional image-based lane detection systems struggle with environmental challenges such as lighting and weather, and the computational demands of deep learning models make real-time performance difficult [1].

The graph in Figure 1 shows the evolution of passive and active car safety technologies over time. Passive safety (red line) includes features that protect occupants during a crash, such as seat belts, airbags, and crumple zones, which saw major development from 1960 to 2000. Active safety (blue line), which helps prevent accidents, started advancing rapidly from the 1990s onward, including systems like ABS (Anti-lock Braking System), ESP (Electronic Stability Program), lane detection, and collision mitigation. By 2020, active safety nearly matched passive safety in significance, with modern technologies focusing on automation, driver assistance, and vehicle-to-vehicle communication for accident avoidance. Lane-departure accidents account for over 50 percent of traffic incidents, highlighting the importance of improving lane detection systems. Traditional lane detection methods based on infrastructure changes or mechanical vision techniques face challenges related to cost, accuracy, and adaptability, especially in complex road environments. Although deep learning-based autonomous driving technologies in countries like the U.S., Germany, and Japan have made significant advancements, real-time performance and accuracy issues persist. The motivation of this research is to develop an enhanced lane line segmentation system capable of handling complex road conditions while maintaining real-time performance, ultimately improving autonomous driving safety and reducing traffic accidents [2].

The increasing number of motor vehicles and traffic incidents has made traffic safety a global priority. Although ITS offer a potential solution for improved driving safety, existing lane detection algorithms struggle under adverse environmental conditions like poor lighting, weather, and road complexity. There is an urgent need for a stable, real-time lane detection system capable of accurately segmenting lanes under such conditions to enhance road safety and reduce traffic accidents [3]. With the rise in traffic accidents, traffic safety has become a critical issue. While intelligent transportation technologies offer a solution, current lane detection systems struggle with performance in complex environments, such as varying lighting, weather, and road conditions. A real-time, high-accuracy vision-based lane detection system is necessary to reduce traffic accidents and improve road safety, supporting the advancement of intelligent transportation technologies [4].

This research addresses several key questions: RQ1 seeks to identify recent algorithms that improve the accuracy and efficiency of real-time lane line segmentation systems. RQ2 focuses on designing a vision-based system for lane departure warnings without expensive hardware. RQ3 aims to determine how an Enhanced YOLOv5 with Seg Head can achieve stable and efficient lane detection despite environmental interference. Correspondingly, the research objectives are RO1, to investigate recent YOLOv5-based algorithms for real-time lane line segmentation, prioritizing accuracy and efficiency in complex environments; RO2, to design a vision-based lane departure warning system using image processing techniques and a priority pixel method, enhancing detection without costly hardware; and RO3, to validate Enhanced YOLOv5 with Seg Head by evaluating improvements in accuracy, efficiency, and optimization in high-error scenarios.

The purpose of this research is to improve real-time lane line segmentation by integrating YOLOv5 with a Seg Head network, enhancing accuracy and reducing processing time. The goal is to optimize the network structure and training algorithm to improve real-time performance and robustness, focusing on pixel-level lane segmentation.

The research presents a YOLOv5-based real-time lane line segmentation system with an integrated Seg Head network for enhanced accuracy and performance. The methodology consists of four stages: model development, evaluation, data preprocessing, and system integration. Public datasets like CULane and TuSimple will be used for preprocessing, including scaling, denoising, and normalization. The architecture features a Spatial Path for detailed feature extraction and a Context Path for multi-scale feature integration, supported by an Attention Refinement Module (ARM). Evaluation metrics include accuracy, recall, mIoU, and frame rate to assess real-time performance. The user interface (UI) is implemented using PyQt5 for real-time interaction.

The research focuses on a YOLOv5-based real-time lane line segmentation system with the Seg Head network, addressing structured and challenging road conditions. However, it is limited to standard driving conditions, with extreme weather not extensively tested. The study does not explore other deep learning models or sensor fusion techniques, and real-time processing may be constrained by computational costs, especially on low-power hardware. This research contributes to real-time, accurate lane detection in autonomous driving by integrating YOLOv5 with a Seg Head network for enhanced lane segmentation. The system is designed to perform effectively under varying road conditions, ensuring high accuracy and real-time processing. The research advances the field of autonomous driving by developing reliable image processing methods suitable for complex environments, improving lane detection systems, and enhancing vehicle safety.

2. Literature Review

Machine vision, as an advanced driver assistance technology, relies heavily on its ability to perceive the surrounding environment. Lane detection based on machine vision is also an image processing technology. How to quickly and robustly obtain lane information is a problem that needs to be solved urgently. Scholars at home and abroad have conducted a large number of studies on this topic and achieved good results. Existing road detection algorithms can be divided into traditional methods and deep learning methods based on the way road features are extracted. On this basis, a new road recognition algorithm is proposed. This algorithm uses the learning ability of deep neural networks to extract features associated with the image and process them. Compared with traditional detection algorithms, deep neural networks can improve their generalization performance by enhancing the training sample set, etc., and therefore have stronger robustness. However, the large number of parameter values and calculations required make higher demands on the performance of computing devices. This paper will provide a review of the application of these two methods in road recognition [5].

For a review of related research, CNKI (China National Knowledge Infrastructure) was my main search engine for locating and accessing literature related to the research of lane line segmentation and real-time detection. In addition to CNKI, academic platforms such as Wanfang Data, VIP Journals, Duxiu, Google Scholar, and IEEE were also referenced to extract relevant research papers. While CNKI and VIP Journal provided a large number of recent Chinese literature on deep learning, YOLO algorithm, lane detection and autonomous driving technology, Google Scholar and IEEE helped to expand the search scope by providing relevant English literature. I also established eligibility criteria to ensure that the articles reviewed were directly relevant to the research. These criteria included: (i) the article must focus on lane line segmentation or detection methods; (ii) the method must involve an artificial intelligence-based model, in particular a deep learning algorithm; and (iii) the article must be written in English or Chinese. Any publications that did not meet these criteria were excluded from review. In addition, the research will give priority to recent research to maintain a current and comprehensive understanding of the field.

The summarized findings of the articles reviewed relating to simple lane line detection scenarios are shown in

Table 1. Summary of related work in simple lane line detection scenarios using computer vision and early AI-based approaches.

Method/Algorithm	Findings/Methods	Limitations/Gaps	Source
Edge Detection and Hough Transform	The system reliably detects multiple lanes in real-time by identifying lane positions and required steering angles through a predictive control framework.	The algorithm faces challenges in complex urban environments with irregular or poorly marked lanes.	(Assidiq, Olifa, & Islam, 2008 [6])
Real-time lane segmentation methods	Discusses methods using both traditional and machine learning approaches for lane segmentation.	Difficulty in maintaining high detection accuracy in environments with heavy traffic or poor road markings.	(Quanguang & Hu, 2009 [7])
Geometrical model and Gabor filter-based lane detection	This method improves robustness in lane detection, especially on highways.	Struggles in highly complex road settings with irregular markings or obstructions.	Shengyan Zhou et al. (2010 [8])
Monocular Vision System	The system is robust against shadows and variations in road textures, and it detects both continuous and dashed lines.	Performance is reduced in extreme weather or irregular lane markings.	(Gupta, 2010 [9])
Edge Detection based on Lane Width + Parabolic Fitting + RANSAC(Random Sample Consensus)	Successfully tracks lanes using grayscale processing and RANSAC-based parabolic fitting.	High computational cost; sensitive to lighting changes and noise; relies on hardware for real-time performance.	(Luo & Qi, 2020 [10])
Multi-condition Lane Feature Filtering + Kalman Filter + Adaptive ROI(Region of Interest)	Accurate and robust multi-lane detection in complex environments; real-time tracking and prediction using Kalman filter.	Sensitive to extreme weather conditions and lighting variations; high computational demands for real-time performance.	(Xuan, Liu, Yuan, Li, & Niu, 2017 [11])
RC-DBSCAN Clustering + Kalman Filter + Feature Fusion	Improved lane detection in complex conditions with better robustness and real-time performance compared to traditional clustering methods.	High computational cost for image processing; sensitivity to lighting and complex road environments.	(Deng, Pu, & Hua, 2021 [12])
Clustering-based Adaptive Inverse Perspective Transformation + Kalman Filter	Improved lane detection accuracy on sloped roads; enhanced robustness with Kalman filter for frame stability.	Computational cost; sensitive to lighting and complex road environments.	(He, Chang, Jiang, & Zhang, 2020 [13])
Randomized Hough Transform (RHT) + Filtering	Improved lane line detection accuracy and reduced false detections using angle-based filtering; enhanced real-time performance through optimized point selection.	Computational cost for complex scenarios; sensitivity to noise and lighting variations.	(Zhu, 2018 [14])
Phase Feature Enhancement + Hough Transform	Effective detection of worn lane markings under low contrast and uneven lighting; improved edge detection accuracy with phase consistency.	Sensitive to noise; requires preprocessing for denoising; high computational demand for phase analysis.	(Chai, 2023 [15])
Feature-based, Model-based, and Learning-based Methods	Summarized the strengths and weaknesses of lane detection techniques, highlighting their progress over 20 years; included comparisons of datasets and performance indicators.	Challenges include variability in road conditions, lighting changes, shadows, and robustness across different scenarios.	(Wu & Liu, 2019 [16])
Machine Vision for Road Marking Damage Detection	Proposed a machine vision method to detect damaged road markings, achieving effective results in simulation tests.	Challenges include handling varying environmental conditions and ensuring real-time detection in diverse road scenarios.	(Ye, 2016 [17])
ROI Selection + Adaptive OTSU Threshold Segmentation	Effective for detecting discontinuous and worn lane markings, enhancing accuracy and robustness in complex environments.	Limited by environmental changes and computational demands for processing adaptive thresholding.	(Guan, Jia, & Gao, 2009 [18])
HSV Image Transformation + Vertical Oblique OTSU	Enhanced real-time lane detection with robust lane identification under varying conditions; improved	Computational complexity due to multiple transformations; sensitive to environmental changes like light-	(Xu & Li, 2023 [19])

below.

Table 1 illustrates that traditional computer vision methods, such as Edge Detection and Hough Transform, perform well in simple scenarios with stable conditions but encounter difficulties in environments with complex lighting, heavy traffic, and poorly marked lanes. Integrating machine learning techniques has led to improvements in accuracy and stability, especially in dynamic settings, though these advancements often require increased computational resources. Early AI-based methods, like grayscale processing with RANSAC, show promise in more challenging conditions but demand significant computational power and remain sensitive to environmental changes. Overall, the studies reviewed highlight the foundational effectiveness of early methods while emphasizing the need for more adaptive and resource-efficient solutions to handle unpredictable road conditions. These observations pave the way for exploring more sophisticated techniques in the following section.

Table 2. Summary of related works in complex lane line detection scenarios using computer vision and advanced AI-based approaches.

Method/Algorithm	Findings/Methods	Limitations/Gaps	Source
RepVGG-A0 + Cross-layer Feature Fusion	Improved lane detection accuracy in complex lane-changing scenarios; achieved high real-time detection speed with 132 FPS (frames per second), up to 220 FPS with TensorRT.	Sensitive to shadows and varying lighting conditions; requires efficient hardware for real-time processing.	(Wang & Huang, 2023 [21])
Fractal Residual Structure + Runge–Kutta Method + Row Anchor Detection	Improved lane detection accuracy and speed in complex traffic scenarios; achieved real-time performance with high accuracy using fractal modules.	Sensitive to complex road conditions and requires preprocessing for optimal performance; computationally demanding.	(Du, Lyu, Wu, & Luo, 2023 [22])
Semantic Information Processing (Segmentation, Fusion, Enhancement, Modeling)	Reviewed 84 advanced algorithms categorized by semantic processing type; provided insights into the strengths and weaknesses of each approach.	Difficulty handling complex environmental challenges and high computational demands for real-time performance.	(Hong & Zhang, 2024 [23])
Dynamic Region of Interest + Ant Colony Algorithm	Enhanced lane detection accuracy under complex lighting by dynamically adjusting ROI and using an ant colony algorithm for edge detection.	Computational cost due to complex transformations; sensitive to extreme lighting variations and noise.	(Liu, Yang, & Wang, 2024 [24])
Two-branch Segmentation Network (LaneNet + H-Net) + Adaptive DBSCAN	Achieved high detection accuracy in complex conditions by using a dual-branch network for semantic and pixel embedding; optimized with adaptive DBSCAN and inverse perspective transformation.	Sensitive to high curvature lanes and computationally demanding for real-time performance.	(Xu, Zhao, Fan, Duan, & Li, 2024 [25])
YOLOP (Panoptic Driving Perception) + Embedded Platform Deployment	Achieved efficient multi-task processing for lane detection, obstacle detection, and drivable area segmentation in real-time; optimized for embedded platforms.	High computational demand; requires significant data and training for accurate performance in diverse conditions.	(Wang, Qiu, Yang, Zhang, & Zhong, 2024 [26])
Improved SCNN + PSA Attention Module + VGG-K Network(Visual Geometry Group)	Achieved enhanced lane detection accuracy in complex conditions using improved SCNN with VGG-K and PSA modules; strong performance on the CULane dataset.	Sensitive to extreme lighting and highly occluded lanes; requires further optimization for real-time deployment.	(Wu, Zhang, Ge, & Yu, 2024 [27])
Instance Association Net (IANet) with Mask-based Feature Separation + Position Encoding	Enhanced low-light lane detection using instance association and position encoding; outperformed existing methods with a 71.9% F1 score in night scenes on CULane dataset.	High computational cost; sensitive to extreme lighting variations and occlusions in complex environments.	(Jiang, Zhang, Dong, Zhang, Wang, 2024 [28])
Multi-scale Feature Fusion + ROI Optimization + Adaptive Thresholding	Improved lane detection accuracy on complex road conditions using feature fusion and adaptive thresholds; demonstrated high robustness in varied weather.	Computationally intensive; affected by extreme lighting and requires high-quality input images for optimal results.	(Yao, 2023 [29])

summarizes advances in lane detection algorithms for complex scenarios. Models like RepVGG-A0 and YOLOP offer high accuracy and real-time performance in multitasking, but are sensitive to lighting and require powerful hardware, limiting their use in dynamic environments. Methods like dynamic ROI with ant colony algorithms and multi-scale feature fusion with adaptive thresholding improve accuracy under variable lighting, but come with high computational costs, making them unsuitable for real-time embedded systems. Techniques like Instance Association Networks and SCNN (Spatial Convolutional Neural Networks) with PSA (Point-wise Spatial Attention module) modules perform well in low-light and occluded conditions but need further optimization to reduce computational demands [20]. Overall, while promising, achieving efficient, real-time lane detection in varied conditions remains a challenge. Future research should focus on improving system adaptability and computational efficiency for stable solutions in autonomous driving.

2.1. Lane Line Detection Using Traditional Computer Vision Techniques

Traditional computer vision techniques like Canny edge detection, Sobel filtering, and the Hough transform were effective in simple environments with stable conditions and clear lane markings. However, these methods struggle with complex road scenarios, dynamic lighting, and non-linear lane markings, leading to reduced accuracy and reliability in modern driving contexts.

• Limitations of Traditional Methods: Traditional methods like Canny edge detection and the Sobel filter perform well in well-lit conditions with clear lane markings, but are highly sensitive to noise and lighting changes, reducing effectiveness in complex environments. The Hough transform is effective for straight lanes but struggles with curved and non-linear lane markings. As such, these methods are less reliable for modern autonomous driving applications, where conditions are more variable.
• Limitations in Complex Environments: Traditional methods fail in complex road conditions like sharp turns, shadows, and low light. Their reliance on fixed algorithms limits adaptability to rapid environmental changes, making them unsuitable for autonomous driving in dynamic conditions.
• Traditional vs. Deep Learning Methods: Traditional methods are efficient in simple environments with low computational demands but lack flexibility in dynamic conditions. Deep learning methods, which can automatically adapt to various environments, offer better performance in complex scenarios but require significant computational resources.

2.2. Advantages of Deep Learning in Lane Detection

Deep learning methods, particularly CNNs, offer improved accuracy and robustness in dynamic conditions, handling varying lighting, road conditions, and perspectives. Lightweight models like YOLO and U-Net enable real-time.

Detection on resource-constrained devices, making them suitable for autonomous driving.

• Main Deep Learning Models: YOLO models, FCNs (fully convolutional network), U-Net, and ResUNet are widely used in lane detection, providing real-time detection and precise segmentation. YOLO is ideal for fast detection, FCNs excel in semantic segmentation, and U-Net and ResUNet are effective in complex road scenarios.
• Summary of Deep Learning Advancements: Deep learning models, particularly YOLO, FCNs, and U-Net, show exceptional performance in lane detection in complex conditions. Despite high computational demands, their accuracy and robustness make them ideal for autonomous driving applications.
• Traditional vs. Deep Learning Methods: Traditional methods excel in simple environments with limited computational resources, while deep learning methods provide better adaptability and accuracy in complex, dynamic conditions but require substantial computational power. Optimizing deep learning models for real-time performance and efficiency is a key focus for advancing autonomous driving technology.

3. Methodology

This study proposes an improved lane line detection method combining YOLOv5 with the Seg head network to enhance performance in real-time lane segmentation. YOLOv5’s efficiency is improved by introducing the Diverse Branch Block (DBB) and YOLACT-based Seg head network, which enhances its ability to detect and segment multiscale lane features, addressing limitations in handling complex lane morphologies. It presents a semantic segmentation network architecture, likely from a paper such as BiSeNet (Bilateral Segmentation Network), designed to achieve high efficiency and accuracy in real-time scene parsing. Here is a breakdown.

3.1. Image Preprocessing

To improve the lane detection system’s robustness, preprocessing methods like histogram equalization, median filtering, and lane line clustering are employed. These techniques address challenges such as noise, lighting, and road conditions, providing high-quality input for detection models.

3.1.1. Histogram Equalization

Histogram equalization enhances image contrast, especially in low-light conditions, but may blur edges under normal lighting. It improves image clarity, making lane boundaries more visible, particularly in images with poorly dispersed pixel values.

3.1.2. Median Filtering

Median filtering removes impulse and scanning noise from images without blurring important details, making it effective for cleaning up noisy lane images, ensuring clearer segmentation.

3.1.3. White and Yellow Lane Splits

The study introduces “priority points” to enhance lane line detection, particularly for white and yellow lanes, by processing these pixels based on their R and G values, improving segmentation efficiency and reducing computational load.

3.1.4. Lane Line Clustering

DBSCAN is used for clustering lane line points, providing flexibility and robustness in noisy environments compared to K-Means, which requires predefined cluster numbers. This clustering approach effectively restores lane line morphology for lane departure warnings.

3.2. Inverse Perspective Transformation

3.2.1. Meaning of the Inverse Perspective Transformation

In this study, the bird’s-eye view image obtained by inverse perspective transformation is utilized, and lane lines are detected in the bird’s-eye view image, which is obtained by inverse perspective transformation, the key to which is to determine a set of corresponding information for inverse perspective transformation. Using the principle of inverse perspective conversion, the mapping relationship between the original image and the inverse stereogram can be obtained by combining the position information of the four points in the original image and the corresponding four points in the inverse stereogram. Here, the inverse stereogram is also called “top view”. The calibration idea proposed in this paper is to consider the vehicle’s path as a perfectly flat geometric plane with infinite extensions, even though in reality, roads in cities are not perfectly flat, as roads with labeled lanes tend to be flatter and curvilinear are less common in cities. Therefore, the simple calibration method proposed in this paper has applications on most urban roads and highways.

3.2.2. Methods of Inverse Perspective Transformation

The transformation from road view plane to image plane is a linear transformation, if the spatial coordinates of the actual road surface are represented by u, v, w, the coordinates of the image are represented by (x, y), and the image imaging space is represented by (x, y’, z’), and the transformation relationship is represented by T, then the transformation formula from road view plane to image video is as follows:

In the transformation matrix TT, the linear transformation is represented by TT, and the inverse perspective transformation is denoted as T-T−1. The image shift is accounted for, with the value of assays set to 1. From the system of linear equations, it is evident that the eight parameters in the transformation matrix can be derived using eight equations. By using values from four points in both (u,v,w)(u, v, w) and (x,y,z)(x, y, z) spaces, a total of eight points from ($u_1,v_1$) to $(u_4,v_4$) and $(x_1,y_1)$ to $(x_4,y_4)$ can be used to compute the stereo transformation of the camera. The goal is to establish the conversion relationship between the road plane and the pixels of the actual road, requiring only four calibration circles. Each calibration circle serves to record a set of corresponding chamfer coordinates. The key aspect of calibration is to fix the camera’s position, angle, and perspective on a flat surface.

Lane line detection involves extracting features of lane lines, which typically have a simple shape and distinct color. A combination of HOG(Histogram of Oriented Gradients) feature extraction and classifiers can often lead to overfitting, especially in grayscale images, which ignore the color information. The lane line’s most prominent feature is its white color, which helps distinguish it from the road. Traditional feature extraction methods may fail to effectively extract the road’s edge. The use of the Hough transform combined with color and edge features offers a more accurate approach for road edge detection. For straight lines, extraction is relatively straightforward, while curves and large curvatures require additional processing. Various vehicle line characteristics, such as positional, shape, edge, and color features, are essential for target recognition and classification. Lane lines, typically white, are easily separated from other road features in color images, making them more recognizable. The effectiveness of vehicle line detection depends on selecting the right characteristics, which directly impacts detection performance. YOLOv5, a convolutional neural network, uses optimization strategies based on the original YOLO architecture. It employs a Focus and CSP structure to extract key information from input samples. The PANet(Path Aggregation Network) structure of YOLOv5 focuses on accuracy, efficiency, and scale, and is used for multi-scale feature layer and prediction. The PANet structure, which is a bottom-up enhancement method based on FPN, combines feature maps from different scales. As shown in Figure 2, top-down fusion involves upsampling the feature map and adding it to the pre-layer feature map via a 1 × 1 convolution. Bottom-up fusion, on the other hand, replaces upsampling with downsampling. The YOLOv5 network, primarily used for lane line detection and recognition, consists of four main modules: input, backend, and frontend. The input side involves preprocessing techniques like Mosaic enhancement, adaptive image scaling, and anchor frame computation. Mosaic enhances detection by randomly scaling and aligning images, which is beneficial for detecting smaller objects. Adaptive scaling reduces images to a fixed size for detection.

In lane line detection, the Hough transform is used for detecting straight lines in images. Whether in raw or bird’s-eye view images, most lane lines can be treated as straight lines. In cases of curves, a bird’s-eye view segmentation makes the lines appear as straight segments, facilitating detection. The Hough transform detects straight lines by extracting boundary pixels from the image, which are then processed to identify lane lines. Figure 2 demonstrates this process, where (a) shows the top view, (b) is the binary edge after Canny edge detection, and (c) displays the result of the Hough transform applied to the edge image.

Lane Departure Warning (LDW) technology is vital for advanced driver-assistance systems. LDW ensures driver safety by accurately identifying the lane position. To minimize false alarms, a fuzzy control algorithm is proposed, which considers vehicle state, mileage, and tire-road distance. The system warns the driver if the vehicle deviates from its lane. The camera’s position, aligned with the vehicle’s center axis, aids in determining the vehicle’s trajectory. By using reverse perspective transformation, the camera’s view is mapped to a bird’s-eye view, and a warning is triggered if the vehicle’s trajectory crosses the lane line.

Region of interest (ROI) detection is an essential strategy in image processing to enhance processing efficiency. In lane line detection, only the road plane is analyzed. Vanishing points on the road can help define the ROI. The haar-texture feature structure can detect vanishing points without the need for camera calibration, although it involves significant computational work. Alternatively, vanishing points can be directly calculated from the camera’s calibration process.

3.3. Improvements to the YOLOv5 Network

3.3.1. Diverse Branch Block (DBB)

YOLOv5 has become a widely used model for real-time lane line segmentation due to its efficient feature extraction and real-time processing capabilities. However, YOLOv5 faces limitations in feature representation and detection accuracy, especially when dealing with multi-scale and diverse lane line structures. To enhance its performance, this paper introduces the Diverse Branch Block (DBB), a multivariate branching module designed to improve feature extraction at different scales and complexities, significantly increasing detection accuracy and robustness. The DBB module is inspired by the Inception network and utilizes a multibranch structure to encode feature information of varying scales and complexities. Traditional convolutional modules typically rely on a single path convolution to extract local features. However, these modules struggle to capture complex structures due to their inability to comprehensively handle multiple scales. In contrast, the DBB module enhances the feature space by integrating multiple convolutional branches of different scales, improving the network’s ability to handle diverse feature complexities. The DBB module consists of four primary branches: a 1 × 1 convolutional branch, a 3 × 3 convolutional backbone, a 5 × 5 convolutional backbone, and an average pooling backbone. The 1 × 1 convolutional branch is used for channel compression and feature fusion, reducing computation and improving feature representation. The 3 × 3 convolutional backbone is optimized for extracting edge and detail information, while the 5 × 5 branch captures broader contextual information, enabling the network to learn long-distance dependencies. The average pooling branch aids in aggregating global information to better understand the overall lane line layout. This multi-branch design allows the DBB module to process multi-scale and complex features simultaneously, improving lane line detection accuracy across various environments and lane line appearances. Furthermore, the module includes residual connectivity, which combines the output feature maps of each branch with the main path through addition. This not only enhances feature fusion across scales but also alleviates the gradient vanishing problem in deep networks. A key feature of the DBB module is its structural re-parameterization. During training, the module functions as a superstructure of multiple branches, while during inference, these branches are compressed into a single convolutional layer to maintain high efficiency and low computational cost. This re-parameterization ensures that the network performs inference at the same speed as the original YOLOv5 model, with the added benefits of the multi-branch structure during training. The integration of the DBB module into the YOLOv5 network involves replacing some of the traditional convolutional layers with the DBB, thereby improving the feature extraction and multi-scale fusion capabilities of the backbone. This integration improves both detection accuracy and flexibility without compromising real-time performance. Compared to the AF(adaptive fusion) method, the DBB(Diverse Branch Block module) achieves significant performance improvements, particularly in feature extraction and fusion for complex lane line structures. It enhances detection robustness by leveraging the multi-branch structure to capture lane line features across various scales in a single pass, ensuring higher accuracy and real-time performance, crucial for autonomous driving systems.

• Detailed Design and Optimization of the Multifaceted Branching Module: The DBB module’s design focuses on both the diversity of the multi-branch structure and its feature extraction capabilities. Each convolutional branch uses a different kernel size to capture features at varying scales: the 1 × 1 branch for channel compression, the 3 × 3 branch for extracting detailed spatial features, the 5 × 5 branch for capturing broader contextual information, and the average pooling branch for integrating global information. The design of residual connections ensures that features from all branches are effectively integrated, forming a richer and more comprehensive representation. Additionally, Batch Normalization (BN) and ReLU activation are used to enhance the feature representation capability, providing the necessary non-linear mappings during training.
• Implementation and Optimization of Structural Reparameterization: The DBB module utilizes structural re-parameterization to convert the multi-branch structure used during training into a single convolutional structure during inference. This method ensures that the network’s efficiency and performance during inference remain consistent with the original YOLOv5 network while benefiting from the multi-branch structure during training.
• Parameter Configuration and Training Optimization Strategies: For optimal performance, several parameter and training strategies were applied to the DBB module. The configuration of convolutional kernel sizes and the number of channels in each branch is carefully adjusted based on the scale and complexity of the lane line features. Additionally, a comprehensive loss function combining cross-entropy loss and Dice loss is used to improve the model’s ability to recognize lane line edges and detailed features. Data enhancement techniques, such as random cropping and rotation, are applied to increase the diversity of the training data and simulate real-world driving conditions. The use of the Adam optimizer and cosine annealing learning rate scheduling ensures fast convergence and stable optimization, further improving the model’s final detection performance. The DBB module’s design and training optimizations ensure its effectiveness in real-time lane line segmentation, delivering enhanced feature extraction, robustness, and computational efficiency.
• Optimization of Training Strategies: The optimization of training strategies includes the use of advanced data augmentation techniques, as well as regularization methods like Dropout and weight decay to prevent overfitting. The optimized learning rate scheduling and use of the Adam optimizer contribute to faster convergence and better model performance.

3.3.2. Improved Feature Fusion Network Architecture Design

To improve YOLOv5’s performance in real-time lane line segmentation, several structural improvements are made to enhance the fusion capability of multi-scale features. These improvements are aimed at increasing the network’s real-time performance and accuracy, particularly in lane line detection. The backbone network of YOLOv5, known for its efficiency in feature extraction, is further optimized by increasing the receptive field and enhancing feature capture through a larger 6 × 6 convolutional kernel. Additionally, the depth and width multiplier parameters are adjusted to balance network efficiency with expressive power. The improved backbone network uses lightweight C3 modules, including the dual-branching C3_DBB module, which enhances the extraction of fine-grained lane line features. The Spatial Pyramid Pooling Fast (SPPF) module is also introduced to improve the network’s ability to perceive features at different scales. The detection head architecture is optimized with modules such as SimFusion_4in, IFM, and InjectionMultiSum_Auto_pool, which enhance multi-scale feature fusion and increase detection accuracy. These modules allow feature maps of different resolutions to interact more efficiently, extracting finer details of lane lines and improving detection performance.

• Integration of the Gather-and-Distribute Mechanism: The Gather-and-Distribute (GD) mechanism is introduced to optimize the fusion of multi-scale features. The Low-GD branch handles small- to medium-sized lane line features, while the High-GD branch processes larger lane line features. The GD mechanism enhances the transfer of information across scales, ensuring the effective fusion of both fine and high-level features.
• Application of Masked Image Modeling Pre-training: To further improve the model’s performance, Masked Image Modeling (MIM) pre-training is employed. This unsupervised pre-training, based on the MAE (Masked Autoencoders) technique, enables the model to learn both global and local image features. The MIM pretraining improves the model’s robustness and generalization ability, enhancing its performance in real-time lane line segmentation tasks.

The improved Seg head segmentation network enhances YOLOv5 for real-time lane line segmentation by adopting key concepts from YOLACT, particularly the separation of prototype mask generation and mask coefficient prediction. This design efficiently handles lane line segmentation through three core modules: the Prototype Mask Generation (PMG) module, the Mask Coefficient Prediction (MCP) module, and the Mask Fusion Module (MFM). The PMG generates a set of base prototype masks that cover various lane line morphologies and scales using multi-scale features and lightweight convolutions. The MCP predicts coefficients that determine the contribution of each prototype mask to the final segmentation, while the MFM fuses these prototypes by multiplying each with its corresponding coefficient to form the final lane line mask. To further improve segmentation, the network incorporates a multi-scale feature fusion mechanism that combines feature maps from different scales. This fusion is guided by a self-attention mechanism, which highlights essential regions and strengthens feature representation for lane line segmentation. The segmentation head’s performance and computational efficiency are optimized by using depth separable convolutions, which reduce the computational complexity compared to standard convolutions, and integrating self-attention to adapt to various lane line scenarios. This modular approach, including structural optimizations, improves YOLOv5’s accuracy and stability for lane line segmentation, ensuring better adaptability for different lane line shapes and scales. The network’s efficiency and flexibility make it suitable for real-time applications while offering valuable design insights for other segmentation tasks.

4. Results and Analysis

The experimental environment for this project is based on a high-performance computing platform with varying hardware configurations. The system includes a 13th Gen Intel(R) Core (TM) i5-13500HX processor running at 2.50 GHz, 32GB of high-speed 4800 MHz DDR5 RAM, and a 2.5 TB SSD. To support complex image processing tasks and model training, an NVIDIA GeForce RTX 4060 Laptop GPU is used. The software environment is Microsoft Windows 10 Professional Edition (version 10.0.19041.867), providing stable operating system support. The development environment utilizes Python 3.7.5 and Anaconda 3.0.0, along with deep learning libraries such as TensorFlow and PyTorch, and scientific computing libraries like Numpy 1.19.2, Scipy 1.5.2, and Matplotlib 3.3.2.

4.1. Training Environment and Evaluation Indicators

This study evaluates the performance improvement of an enhanced algorithm using mAP@0.5 and Recall (R) as evaluation metrics. Recall (R) is the proportion of correct predictions out of all true targets and is mathematically expressed as: Recall = TP/(TP + FN). Here, TP, FP, FN, and TN represent the number of true positive, false positive, false negative, and true negative predictions, respectively. Average Precision (AP) is a key indicator of model superiority in a specific category, calculated as: AP = 01P(R)dR. Mean Average Precision (mAP) represents the average merit of the model across all categories and is computed as the average of all APs. The mAP@0.5 is used when the Intersection over Union (IoU) threshold is set to 0.5. IoU is calculated as the area of overlap between the predicted and real frames divided by the area of their union, as expressed by: IoU = S_overlap/S_union. The mAP is calculated as: mAP = (∑(i = 1)NAP_i)/N

4.2. Analysis of Experimental Results

This section provides a detailed examination of the performance of the enhanced YOLOv5 network in real-time lane line segmentation tasks, including results from ablation tests and side-by-side comparison experiments. The improvements introduced to YOLOv5 will be assessed based on several performance criteria.

• Analysis of Ablation Experiment Results: The ablation experiment evaluates the contribution of each module to the overall model performance.

  Table 3. Comparison of traditional computer vision and deep learning methods.

  Comparisons	Traditional Computer Vision Methods	Deep Learning Methods
  Applicable Environment	Simple, stable environments.	Complex, dynamic environments.
  Adaptability and Extension	Limited flexibility, difficult to adapt to diverse settings.	Highly flexible and can be adapted to a range of settings.
  Computational Resource Demand	Low resource demand, suitable for limited hardware.	High resource demand, usually requires GPU support.
  Performance	Stable in simple scenarios, reduced accuracy in complex conditions.	High accuracy and robustness in complex, dynamic conditions.

  displays the performance metrics for four models: the original YOLOv5, YOLOv5-DBB, YOLOv5-GOLD, and YOLOv5-mix. The baseline YOLOv5 model achieves an F1 score of 0.821, precision of 0.904, recall of 0.753, and mAP@0.5 of 0.789, providing a stable reference for subsequent improvements. The YOLOv5-DBB model, which introduces the Diverse Branch Block (DBB), slightly improves precision and mAP@0.5, but recall remains unchanged, demonstrating that DBB enhances multi-scale feature extraction while having minimal impact on recall. YOLOv5-GOLD, which incorporates an improved segmentation head, improves recall significantly to 0.765, though precision slightly decreases to 0.897, and mAP@0.5 rises to 0.803. The YOLOv5-mix model, which combines both DBB and the improved segmentation head, further optimizes performance, increasing recall to 0.758 and mAP@0.5 to 0.804 while maintaining high precision (0.907). The results confirm that the DBB module improves precision and mAP, while the segmentation head enhances recall and segmentation accuracy.
• Analysis of Results from Side-by-Side Comparison Experiments: In side-by-side comparison experiments, the improved YOLOv5 model was compared with other mainstream models, including LSKNet and VanillaNet.

  Table 4. Results of ablation experiments.

  Network Models	F1	Precision	Recall	mAP@0.5
  yolov5	0.821	0.904	0.753	0.789
  yolov5-DBB	0.822	0.906	0.753	0.795
  yolov5-GOLD	0.825	0.897	0.765	0.803
  yolov5-mix	0.825	0.907	0.758	0.804

  shows that YOLOv5 outperforms both models across all metrics. LSKNet shows significantly poorer performance with an F1 score of 0.497, precision of 0.756, and recall and mAP@0.5 of 0.371 and 0.300, respectively. VanillaNet performs better than LSKNet but still lags behind YOLOv5, with an F1 score of 0.595 and mAP@0.5 of 0.390. In contrast, the improved YOLOv5-GOLD and YOLOv5-mix models significantly outperform LSKNet and VanillaNet, excelling in precision, recall, and mAP@0.5. These results validate the effectiveness of the DBB and improved segmentation head in enhancing both the precision and recall of YOLOv5, making it highly competitive in real-time lane line segmentation tasks.
• Specific Analysis and Discussion: The introduction of the DBB module enhances the model’s ability to extract multi-scale features, improving both accuracy and mAP. This allows YOLOv5 to handle lane lines of varying sizes and complexities more effectively, contributing to its robustness. The Seg head segmentation network, based on the YOLACT architecture, improves recall and segmentation quality by efficiently generating accurate lane line masks. The integration of a multi-scale feature fusion mechanism, using self-attention, further enhances the model’s ability to capture fine-grained features and structural information. These improvements, combined with the results from comparison experiments, confirm the superior performance of the enhanced YOLOv5 model in real-time lane line segmentation.

As shown in Figure 3, the YOLOv5-DBB model introduces the Diverse Branch Block (DBB) on top of the baseline model. With this multibranch structure, YOLOv5-DBB slightly improves the F1 score to 0.822, the precision to 0.906, and mAP@0.5 to 0.795, while the recall remains unchanged (0.753). This result indicates that the DBB module improves the overall precision and average accuracy of the model by enhancing the network’s ability to extract multiscale features, but has less impact on the recall. This is in line with the original design intent of the multivariate branching module aimed at enhancing feature representation.

As shown in Figure 4, the YOLOv5-GOLD model further introduces an improved Seg head segmentation network based on YOLOv5-DBB. Compared to YOLOv5-DBB, YOLOv5-GOLD improves the F1 score to 0.825, the precision slightly decreases to 0.897, but the recall significantly improves to 0.765, and mAP@0.5 improves to 0.803. This change shows that the improved Seg heads, by capturing the lane line boundaries and details more accurately, significantly improves the recall of the model and the overall average precision, even at the cost of slightly lower precision. This balance is particularly important for practical applications to avoid missing lane lines.

As shown in Figure 5, the YOLOv5-mix model combines the DBB module and the improved Seg head, and its performance is further optimized on top of YOLOv5-GOLD, with the F1 score remaining at 0.825, the precision rate improving to 0.907, the recall rate increasing to 0.758, and mAP@0.5 reaching 0.804. This shows that the combination of the multivariate branching module in combination with the improved segmentation head can further improve the recall and average precision without sacrificing the precision rate, achieving an overall optimization of performance. The design goal of the UI system is to provide an intuitive and efficient interaction platform to help users load video data, run lane line segmentation models, display segmentation results in real time, and evaluate and analyze the results, the UI interface written in this paper is shown in Figure 6.

5. Conclusions and Future Work

This paper improves the YOLOv5 network for real-time lane line segmentation by introducing two key innovations: the Diverse Branch Block (DBB) and an enhanced Seg head segmentation network based on YOLACT. The DBB module improves feature extraction across multiple scales and complexities, boosting YOLOv5’s detection accuracy and robustness.

The Seg head, which separates mask generation from coefficient prediction, improves segmentation efficiency and accuracy by utilizing multi-scale feature fusion. Experimental results show that the enhanced YOLOv5 outperforms other models in key metrics, such as precision, recall, and mAP@0.5, significantly advancing real-time lane line segmentation.

• Optimization of DBB Module: Further refine the Diverse Branch Block (DBB) to enhance feature representation and overall model accuracy while maintaining efficiency.
• Multi-modal Data Fusion: Incorporating sensor fusion can improve segmentation performance across varied conditions.
• Robustness in Extreme Conditions: Further work is needed to optimize the model’s ability to handle severe weather and lighting conditions.

This study enhances YOLOv5’s real-time lane line segmentation capabilities, paving the way for further research on optimizing the model’s robustness, adaptability, and efficiency in diverse road environments.