1. Introduction
The automotive industry is currently experiencing strong growth. Factors behind this are increasing congestion, environmental issues, wireless network expansion, efficiencies in automotive logistics, and demand for driver, passenger, and pedestrian safety. In that sense, advanced driver assistance systems (ADAS) have shown a very significant development since the early 1970s. ADAS focuses on vision-based applications, with examples including lane departure warnings, lane keep assist, pedestrian detection, and collision avoidance. Although recent technological advancements show significant development results, there are still problems to solve. For example, actively-researched problems include gauging uncertainty in a static/dynamic environment, improving processing time response, and managing road complexity.
Vision-based lane marking approaches focus primarily on the color-pattern changes between the road surface and pavement marking materials [
1,
2,
3,
4,
5,
6]. This difference in color is used to detect lane markings by extracting the features, such as edge and color information. Once the lane marking features are extracted, a model fitting is implemented taking into consideration the lane marking geometry design, specifically width and length. Last but not least, processing time plays an important role in decision making. By considering color-based methods, researchers in [
1,
2,
3] focused on the task of lane marking detection using the HSI (hue, saturation, and intensity) color model. The algorithmic strategy proposed by the authors [
1,
2] shows good performance in detection and processing time. However, this algorithm extracts the lane marking using only the color information belonging to the markings, which is not accurate enough considering that there is a wide range of traffic signals on the road with the same color information (crosswalk, stop lines, one-way streets, merging/diverging traffic, etc.) without also taking into consideration the noise information (spilled paint on the road or surface with colors that belong to the lane marking). Researchers in [
3] proposed a high-precision lane-level localization using a stereo camera. After the rectification between stereo images is finished, the algorithm uses the intensity value from HSI color space, and then the edge information is extracted. Unlike the previously mentioned research, these authors proposed additional detectors in order to avoid false detection (humps, crosswalks, traffic word signals, arrows). Once the edges are extracted, the authors proposed a hyperbola fitting to search the pixels belonging to the lane marking within the image. To solve vehicle localization problems, the authors use the random sample consensus (RANSAC) algorithm. Although the algorithm provides a good estimate of reliability due to the developed strategy, integration of these steps is time consuming.
Researchers in [
4,
5] give examples of edge-based methods. Mammeri et al. [
4] proposed a lane marking localization by combining the maximally stable extremal region (MSER) and Hough transform (HT). MSER is used to extract the stable region given by lane markings, traffic signs, etc. Once the regions are extracted, the algorithm refines the previous results. Here, the task entails line detection using a probabilistic progressive Hough transform to extract the line segments with a minimum required length. Then, the line segments within 10
and 85
are considered lane marking candidates, and the rest of the segments are removed. The proposed method shows reliable results; however, given that they depend on the environment, the algorithm’s response is not stable (Tables 3 and 4 in [
4]). Additionally, the strategy is only suitable for a straight lane marking regions with a low velocity and high camera frame rate constraints. Researchers in [
5] proposed an efficient lane detection method using spatiotemporal images. After the alignment between the frames is performed, the lane points are detected using the most dominant straight line defined by the HT. Then, a weighted least-squares fitting is used as a fitting model. The algorithm has good performance with high processing time. Similarly, as gauged by authors in [
1,
2], the algorithm shows some error in regions such as bumps.
Researchers in [
7] introduced a laser scanner application to estimate the vehicle heading angle. To do that, the authors proposed identifying the lane within the road surface by finding differences between road and marking materials. Consequently, the lane region, the lane marking, the centroid of the lane width, the road geometry as well the vehicle heading angle are iteratively estimated. Firstly, for a laser line-scanner input range data, the set of points within the region of interest is extracted. The region of interest is defined as 20 m × 4 m, the square box is located 10 m to the left and right of the laser sensor and between 10 m and 14 m in front of the laser. Secondly, in order to detect the road surface, an unsupervised density-based spatial clustering is implemented. As a result, the number of lanes within the road are detected. Thirdly, the lane markings are detected to improve the accuracy of the previous step. This is done by finding the discontinuity caused by the change between surface, road to lane marking or vice-versa. Fourthly, the centroid of the lane is the center point computed as the difference between the left and right lane marking located ahead of a vehicle (lane width). Fifthly, using the centroid information, the road geometry parameters are extracted; the deflection angle
θ, the chord length d
, and the vehicle heading angle
δ are computed. Although the algorithm shows a good performance and accuracy, it may encounter problems in regions where lane markings are considerably deteriorated.
All the research introduced above shows reliable results for use in the field of advanced driver assistance systems or autonomous navigation. However, all of these methods are still facing problems, either in processing time or performance. Consequently, in order to improve the current algorithms, a vision-based real-time lane detection algorithm is proposed for application in intelligent vehicles. In that sense, the presented method does not just take into consideration the features of color, edge, and length lane marking information, but also combines them, resulting in a speed-dependent, color-edge hybrid lane marking detection algorithm.
To summarize, the main steps of the proposed method are described below. Regions within the image are classified into categories of collision risk, zero collision risk, and regions above the horizon. The proposed algorithm focuses primarily on the collision risk region, which is extracted taking into account the vehicle speed (considered as an important key), and the region of interest (ROI) increases as the speed increases and vice versa. The color and edge lane marking features are extracted by analyzing the color and width information. Then, by combining both features, the midpoints of the lane markings are extracted. This process plays an important role, and, because of this strategy, the midpoints are extracted if they belong to lane marking types; otherwise, no action is carried out. In essence, other marking types such as crosswalks, line stops, and traffic signals on the road surface are rejected. The algorithm is not sensitive to these types of marking. This strategy reflected a substantial decrease in processing time compared with the current lane marking algorithms, in which additional processing should be performed in order to remove the noise given by different types of markers. Then, using the midpoint features, a hierarchical fitting model is implemented. A lane marking detection algorithm that solely relies on a line fitting model is not strong enough due to various road environment scenarios. On the other hand, based on the processing time as well as the road geometry design, implementation of a polynomial fitting model along the vehicle trajectory is not the best solution. Using the proposed sensor setup, it was observed that around 5% of the lane markings within the image can be defined using a curve fitting model, while a line fitting model should be used for the rest. Consequently, to solve this problem, a hierarchical line/curve fitting model has been proposed. These strategies allow the algorithm to avoid the use of expensive computation when it is not required. Finally, the lane region is estimated by using the line/curve fitting parameters.
2. System Overview
Figure 1 and
Figure 2 show the sensor configuration used to gather the road imagery data, as well as the vehicle platform. The camera is located at a height of 1.8 m and tilted down at an angle of 9
with respect to the horizontal projection, 2.3 m from the vehicle front bumper, and 0.5 m to the right of the vehicle centerline.
Assuming the vehicle undergoes planar motion, the geometry of the model is described as follows: h is the height of the camera above the ground, d is the distance between the camera at ground level and the point in the real world, is the point projected to the center position of the image plane through the optical axis of the camera, is the centroid of the lane at the ground level, γ is the angle between the optical axis of the camera and the horizontal projection, α is the angle formed by scanning the image from top to bottom with respect to the center point of the image , is the pixel position located at , is the pixel located at the center of the image plane, is the focal length in pixels, is the focal length in millimeters, is the image width in pixels, and is the CCD or CMOS image sensor width in millimeters. The vehicle data were acquired by a National Instrument PXI Express module (National Instruments Corporation, Austin, TX, USA). Additionally, a laser measurement system (LMS) was mounted on the vehicle centerline at a height of 1.7 m and tilted down at 7.5 with respect to the horizontal projection.
Contrary to other studies [
4,
6], the extracted ROI is vehicle speed-dependent. To extract the ROI, the typical stopping distance is used (according to the Department for Transport and Driver and Vehicle Standards Agency of the Government of the UK). The purpose of defining the ROI using this strategy is two-fold: reducing the processing time using a dynamic speed-dependent ROI and determining the region for collision avoidance.
Figure 3 shows the road region described using the proposed sensor setup and the vehicle speed. From the sensor setup, the region over the horizon is defined (region over the red line in
Figure 3). The zero collision risk region is defined as the region between the horizon and the collision risk region (between the red line and the middle green line in
Figure 3). The collision risk region is extracted using the vehicle speed (green box in
Figure 3). Hence, the region is defined by using a model-based typical stopping distance [
8].