Robust Lane Detection and Tracking Algorithm for Steering Assist Systems

Abstract

Modern vehicles rely on a multitude of sensors and cameras to both understand the environment around them and assist the driver in different situations. Lane detection is an overall process as it can be used in safety systems such as the lane departure warning system (LDWS). Lane detection may be used in steering assist systems, especially useful at night in the absence of light sources. Although developing such a system can be done simply by using global positioning system (GPS) maps, it is dependent on an internet connection or GPS signal, elements that may be absent in some locations. Because of this, such systems should also rely on computer vision algorithms. In this paper, we improve upon an existing lane detection method, by changing two distinct features, which in turn leads to better optimization and false lane marker rejection. We propose using a probabilistic Hough transform, instead of a regular one, as well as using a parallelogram region of interest (ROI), instead of a trapezoidal one. By using these two methods we obtain an increase in overall runtime of approximately 30%, as well as an increase in accuracy of up to 3%, compared to the original method.

1. Introduction

Lane detection and tracking are challenging problems in computer vision as developing a robust and computationally efficient algorithm is not a simple task. Figure 1 and Figure 2 show some factors that can cause inaccurate results. Methods that can adequately detect and track lanes must also be optimal. Otherwise, they may not be suitable for practical implementations.

Different lane detection and tracking algorithms will be presented in this section, while in the following section we will analyze one mentioned technique in detail. Then, we will study its computational complexity and disadvantages. In the end, optimization ideas will be introduced, and results will be compared to the primary method.

M. Aly [1] proposes a method for detecting lanes in urban roads. Inverse perspective mapping is used to generate a top view image of the roadway. The image is then filtered using a two-dimensional Gaussian kernel. Finally, the Hough transform is used for detecting straight lines, while the random sample consensus (RANSAC) method is performed for line fitting. As mentioned in [2], detection is not proper in the presence of crosswalks or nearby vehicles.

The ant colony optimization algorithm (ACO) is suggested by P. Daigavane and P. Bajaj in [3] for line detection. The first step is to resize the original input image to reduce computational time. The median filter is applied for noise filtering on a converted grayscale image, then edge detection is performed by the Canny edge detector. The resulting image is then used by the ACO algorithm for generating edge information that was lost in the edge detection process. The final step is to detect straight lines using the Hough transform.

Z. Kim [4] presents a robust lane-detection-and-tracking algorithm to deal with challenging scenarios such as lane curvature, worn lane markings, lane changes, and emerging, ending, merging, and splitting lanes. He firstly uses gradient and intensity bump detectors to remove non-lane markers. He then uses a real-time lane-marking classifier and groups the lane markings into lane-boundary hypotheses, which he evaluates and groups based on a probabilistic framework. Particle filtering is used for lane tracking. As mentioned in [2], the algorithm is robust in the case of a straight or curved road.

Borkar, Hayes, and Smith [5] propose a method based on the Hough transform and iterated matched filters. The first step is to convert the image to grayscale and apply the Inverse perspective mapping. An adaptive threshold technique is performed to obtain a binary image which is then split into two halves, one for the left lane marker and the other for the right one. To find their approximate center, a matched filter is applied at each point along the line, detected using Hough transform. Finally, after the center is estimated, the RANSAC algorithm is applied for lane detection. A Kalman filter is also used to track the lines.

The authors of [6] developed a lane detection method suitable only at night. First, the algorithm defines a region of interest to preserve only the road pixels. Next, the image is converted to grayscale and adaptive thresholding is employed for extracting bright objects. The output image is then divided into two halves, each containing one lane marker. The Hough transform is finally applied for detecting straight lines.

A. Sharma and M. Kumar [7] propose a basic algorithm implemented using the Open Computer Vision (OpenCV) library in Python. Firstly, the method converts the input images to grayscale and applies the Canny operator for finding edges. Next, a trapezoidal region of interest (ROI) is used to eliminate sky and side road objects, but it does not eliminate the area between the lane markers, which also does not interest us and is therefore a downside. The Hough transform is then used to detect lines but using it for each edge pixel in every single frame for finding lane markers is a redundant process and may also be a source of incorrect detection. The RANSAC modeling tool is used to clean datasets from noise. In this paper, we will explore this algorithm in-depth and propose improvements, such as using a parallelogram ROI and applying a probabilistic Hough transform.

J. Cao et al. [8] propose a lane detection algorithm for intelligent vehicles in complex road conditions and dynamic environments. They convert the distorted image and use the superposition threshold algorithm for edge detection to obtain an aerial view of the lane via a region of interest extraction and inverse perspective transformation. They use the random sample consensus algorithm to fit the curves of lane lines based on the third-order B-spline curve model, evaluate the fitting of the curve, and calculate its curvature radius.

Q. Li et al. [9] propose a robust and efficient method to expand the application of vision-based lane-detection methods to cover low-speed environments. They initialize the reliable region near the vehicle and dynamically construct a series of rectangular detection regions along the road. They extract the edge points of the lane markings based on accurate marking width limitations using improved symmetrical local threshold edge extraction. They propose a novel Bresenham line voting space to improve the process of line segment detection. Combined with straight lines, polylines, and curves, the proposed geometric fitting method has the ability to adapt to various road shapes. They use different status vectors and Kalman filter transfer matrices to track the key points of the linear and nonlinear parts of the lane.

X. Zhang et al. [10] propose an adaptive lane feature learning algorithm that can automatically learn the features of a lane in various scenarios. They construct a two-stage learning network based on YOLO v3 (you only look once, version 3) with the structural parameters of the YOLO v3 algorithm modified to make it more suitable for lane detection. They propose a method for automatic generation of the lane label images in a simple scenario, which provides label data for the training of the first-stage network. They use an adaptive edge detection algorithm based on the Canny operator to relocate the lane detected by the first-stage model. They use the images processed by the above method as label data for the training of the second-stage model.

T.M. Hoang et al. [11] propose a method to overcome various illumination problems, particularly severe shadows, by using fuzzy system and line segment detector algorithms to obtain better results for detecting road lanes by a visible light camera sensor. T.M. Hoang et al. [12] propose a method for road lane detection that distinguishes between dashed and solid lanes.

2. Materials and Methods

2.1. Lane Detection Algorithm

In this section, we will analyze the method proposed in [7] due to its simplicity and facile implementation using the OpenCV library. The most important techniques mentioned in the paper will be detailed in the following paragraphs. Moreover, the results will be used for implementing a simple steering assist system. Results and improvements will be introduced in the next sections.

2.1.1. Image Filtering

To accurately detect lines, it is essential to preprocess the image by using a noise-removing filter. For our purpose, one approach is to blur the image using a Gaussian filter.

Blurring the image is achieved by the convolution between the original image and a kernel that contains Gaussian values. The 2D Gaussian function is expressed using the following equation:

$$G\left(x, y\right)=\frac{1}{2\pi {\sigma }^2}{e}^{-\frac{x^2+y^2}{2{\sigma }^2}}$$

(1)

In the above equation, x and y inputs are distances from the origin on the horizontal and vertical axis, respectively, while σ represents the standard deviation of the Gaussian distribution. We can observe a graphical representation of the Gaussian function in Figure 3. Figure 4 shows an example of blurring using a 3 × 3 Gaussian kernel by computing for each pixel the scalar product between the kernel and a sliding window with the same size as the kernel and centered on the pixel (the matrix is padded by border mirroring if the sliding window overflows the image boundaries).

2.1.2. Canny Edge Detector

Canny edge detector [13] is an algorithm developed by John F. Canny that can significantly reduce the number of erroneous edges detected in an image.

After applying a Gaussian filter, as described in the previous chapter, four filters are used to detect vertical, horizontal, and diagonal edges. The values for the first derivative in the vertical and horizontal directions are obtained using an edge detection operator. Using the derivatives, we can compute the edge gradient (2) and direction (3). Finally, four angles, 0°, 90°, 45°, and 135° are used to describe the horizontal, vertical, and both diagonal directions.

$$G=\sqrt{G_x{}^2+G_y{}^2}$$

(2)

$$\theta =atan2\left(G_x+G_y\right)$$

(3)

The following process of the algorithm is edge thinning. In the gradient image, we compare the pixel’s edge strength with the edge strength of the neighbor pixels in both the negative and positive gradient directions. The value will be preserved only if its edge strength is the highest compared to the neighbor pixels.

For example, if there is a vertical edge like the one in Figure 5, the red-framed pixel will be compared with the two black-framed pixels on the left and right.

At this point, the real edges are more accurately emphasized, but there may still be false edges generated by color variation or noise. Therefore, two threshold values are required to classify the remaining edge pixels.

An edge pixel is classified as strong if its gradient is higher than the highest threshold value. On the other hand, if its gradient is lower than the other threshold value, it is suppressed. The pixels that have a gradient value between the two thresholds are classified as weak.

Finally, we can conclude that detected strong edge pixels come from the real edges, whereas weak edge pixels are uncertain because they can be related to noise or color variations. Usually, weak edge pixels are connected to strong edge pixels if they come from real edges. Tracking edge connections can be made using connected-component labeling techniques.

Applying the Canny edge detector algorithm on a frame image produces the result shown in Figure 6.

Our purpose is to track edges that are part of the road so we can define a trapezoidal region of interest. We can see that the Canny edge detector outputs a binary image, therefore we can use the bitwise and operator to keep edges only in the defined region. Figure 7 shows the final post-processed image.

2.1.3. Hough Transform

The Hough transform [14] is a common technique, used in computer vision, for finding imperfect instances of objects such as lines, circles, or even other arbitrary shapes. It is based on a voting procedure, performed in previously-defined parameter space, called accumulator space, from which most voted candidates are selected.

As the lane markers are straight lines in each image frame, we would use Hough transform to detect those in the defined region of interest. A straight-line y = mx + n becomes a single point P(n, m) in the parameter space. However, using the above equation is unsuitable for vertical lines where the slope parameter m is not finite. Thus, we will need another way to represent lines, and the Hesse normal form is the best alternative (4).

$$\rho =xcos\theta +ysin\theta$$

(4)

where $\rho$ is the length of the perpendicular from the origin to the line and $\theta$ is the angle between $x$ axis and the perpendicular. (Figure 8).

Using the above equation, it is possible to associate a line with a ($\rho , \theta$) pair in a 2D array, commonly named the Hough space. For each edge point in the image, the set of all lines passing through that point is related to a single sinusoidal curve in the accumulator. Therefore, the problem of detecting collinear points becomes the task of finding concurrent curves. Figure 9 shows 4 collinear points p0, p1, p2, and p3 with their corresponding curves c0, c1, c2, and c3. The 4 curves intersect in one point ($\rho , \theta$) which is the pair value that describes the line.

The first step of the line detection algorithm is to decide on the values of $\rho$ and $\theta$. Often, as mentioned in [15], the range of $\theta$ is from 0 to 180 degrees and $\rho$ is between −d and d where d is the length of the image’s diagonal. We will need to quantize the range of $\rho$ and $\theta$ so the possible number of values is finite.

As we have mentioned before, the Hough transform is based on a voting procedure, so it requires a 2D array representing the Hough space accumulator. The size of the array should be the number of possible values of $\theta$ times the possible values of $\rho$. All accumulator bins must be initialized to zero.

For every edge pixel, loop through all the possible values of $\theta$ and calculate the corresponding $\rho$ using (4). The next step is to increment the accumulator value for each ($\rho , \theta$) pair index.

Finally, loop through all the values in the accumulator and select those which are larger than a certain threshold. Having the two-line parameters $\rho$ and $\theta$, we can switch back to the form of y = mx + n if needed.

Figure 10 shows one frame image after applying the Canny edge detector, while Figure 11 is the associated Hough space. We can notice two main bright spots areas corresponding to the lane markers.

2.1.4. Detecting Lanes

We have concluded that with the help of the Hough transform we are able to detect lines in the region of interest. The first step is to classify those lines so that we can determine which one of them is the left or right lane. A simple and intuitive method is to calculate the slope for each line. Because of the camera view perspective, if the slope is negative then the line may be the left lane, whereas a positive slope indicates a possible right lane.

The next step is to calculate an average line for each lane side. This can be performed by averaging the x and y coordinates of the points that define our detected lines. We can see the detected lanes highlighted in Figure 12.

At this point, in order to detect the turn and provide the necessary indications, we can analyze the slopes of the final lines. It should be noted that in the case of a left turn, the slope of the right line is the one that varies the most. Similarly, in the case of a right turn, we will study the slope change of the left line.

Finally, we can choose a threshold value to classify some basic states like going forward, turning left, or turning right. Several such thresholds can be chosen to provide more precise indications depending on the road curvature.

For a graphical illustration, a turn sign is added between the two lanes (Figure 13).

2.2. Correctness and Complexity

2.2.1. Correctness

The designed algorithm in [7] provides good assistance in most situations, but there are cases in which the accuracy of indications can be improved, or it is even completely wrong.

The issues that may occur are mostly because of the line detection process. In each frame image, the algorithm may provide lines that are not of interest as they are not the actual lanes. Therefore, it is necessary to reject them because they will decisively contribute to the calculation of the final line slope and intercept.

The most common objects that generate fake lanes are parapets, shadows, road ground signs, or even a close car that enters the region of interest. We note some examples of fake lanes after applying the Canny Edge detector in Figure 14 and Figure 15.

2.2.2. Space and Time Complexity

For each frame image, the algorithm must filter, detect edges, and extract lines from it. For the first two operations, the time complexity is O(n log₂(n)), where n is the number of pixels in the image obtained by multiplying its length and height.

The Hough transform for line detection requires an additional memory of k², where k is the number of bins chosen. The temporal complexity depends on the quantization and the number of edge points in the region of interest.

An extension of the Hough transform [16] involves using the gradient to avoid searching lines for certain angles. The next section will provide new observations that will lead to computational improvements of the method.

2.3. Improving the Method

2.3.1. Probabilistic Hough Transform

Even for line detection, the Hough transform requires a lot of computation to be done. One way to reduce the number of computations needed would be to reduce the number of edge pixels that need to be considered as potential line candidates. Kiryati et al. [17] introduced the idea of the probabilistic Hough transform, which aims to do exactly that. By selecting a random subset of edge pixels, from our initial set, we linearly reduce the time based on the number of pixels we selected. Afterward, the Hough transform can be applied to detect lines, the idea being that a certain subset of pixels still retains representative enough features to detect the lines we are interested in. Obviously, finding the number of pixels that are still representative enough to retain the main features we are interested in, as well as to reduce the computational time noticeably, is not an easy task, and varies from problem to problem. In our case, the threshold was chosen by trial and error, aiming at both accuracy and computational efficiency.

Figure 16 shows the Canny edge detector output image where random edge pixels have been removed. We can see that lane shapes have been preserved and, even better; many lines that we are not interested in have been eliminated.

Unfortunately, if the edge pixels subset is too small, we may lose line shapes. Figure 17 compares the accumulation buffers for the original Hough transform versus the probabilistic Hough transform ones. We can see clearly that the second one has fewer bright areas meaning that less data has been processed.

We can also notice that the two main areas associated with the two road lanes have been preserved using the probabilistic Hough transform.

2.3.2. Defining a Dynamic Parallelogram Region of Interest

The Hough transform is a great tool for detecting lines in an image, but using it for each edge pixel in every single frame for finding lane markers is a redundant process and may also be the source of incorrect detection.

An important observation is that, from one frame to another, the lane lines do not substantially change their slope or y-intercept. Thus, we could significantly reduce the search space size since we will approximately know where the lane lies in the next frame.

We suppose that in the current frame we have detected the real lane. OpenCV [18] functions like HoughLinesP return the coordinates of the two endpoints for a detected line. Using those points we can define a parallelogram region of interest so that every single edge point that does not lie in the area will be suppressed in the next frame.

Parallelogram pixel length should be chosen depending on the recording resolution, recording frame rate, and also lane width. As we do not know if the road turns left or right, the detected line should pass through the center of the parallelogram. (See Figure 18, where the red line is the detected lane and the green parallelogram is the dynamic region of interest where Hough transform will search lines in the next frame).

The same process should be applied to the right lane. Using the described method, we have optimized the pixel edge search space and rejected some of the lines we are not interested in.

To create the dynamic parallelogram region of interest, let us suppose there are A(xa, ya) and B(xb, yb), the two endpoints’ pixels of the detected line, where yb > ya. For each y between ya + 1 and yb − 1 use Equation (5) to calculate the corresponding x value. The final x value is then rounded and each n pixels to the right and left are set to 1, where n is half of the parallelogram length.

$$x=\frac{\left(y-y_a\right)\left(x_b-x_a\right)}{y_b-y_a}+x_a$$

(5)

Figure 19 compares the edge pixels in the trapezoidal region of interest versus edge pixels preserved using the two parallelogram masks around the lane markers starting from the same input frame.

However, creating a parallelogram region of interest image and then applying the and bitwise operator is not the most optimal approach. The purpose is to find only a simple condition that will tell us if a pixel lies in the parallelogram area without creating an actual mask. There are many proposed methods [19], but we are interested in a solution that requires as few computations as possible.

Suppose we have a parallelogram spanned by two vectors $\overrightarrow{PQ}$ and $\overrightarrow{PR}$ where $P\left(xp, yp\right)$, $Q\left(xq, yq\right),$ and $R\left(xR, yR\right)$ are corners. We want to check if point $A\left(xa, ya\right)$ lies inside the parallelogram (Figure 20). $\overrightarrow{PQ}$ is the vector determined by the two endpoints of the line whereas $\overrightarrow{PR}$ is a horizontal vector having a magnitude equal to n where n has the same meaning as previously specified.

We can write $\overrightarrow{PA}$ using two vectors parallel to $\overrightarrow{PQ}$ and $\overrightarrow{PR}$:

$$\overrightarrow{PA}=n\overrightarrow{PQ}+m\overrightarrow{PQ}$$

(6)

To calculate n and m we will use the determinant formula:

$$det\left(\overrightarrow{v}, \overrightarrow{w}\right)=det\left[\begin{array}{cc}{v}_1& w_1\\ v_2& w_2\end{array}\right]= v_1 w_2- v_2 w_1$$

(7)

where $\overrightarrow{v}=\left[v_1 v_2\right]$ and $\overrightarrow{w}=\left[w_1 w_2\right]$

$$n=- \frac{det\left(\overrightarrow{PA}, \overrightarrow{PQ}\right)}{det\left(\overrightarrow{PQ}, \overrightarrow{PR}\right)}$$

(8)

$$m= \frac{det\left(\overrightarrow{PA}, \overrightarrow{PR}\right)}{det\left(\overrightarrow{PQ}, \overrightarrow{PR}\right)}$$

(9)

If n (as in Equation (8)) and m (as in Equation (9)) are both in [0;1] then point A is inside the parallelogram defined by P, Q, and R.

The next step is to apply the Hough transform for each edge pixel that meets the condition stated above.

2.3.3. False Lanes Rejection

As we saw in Section 2.2, there are cases in which detected lines are not the actual lane markers. Therefore, we need some methods through which we can reject those.

An intuitive observation is that lanes have a value slope close to 1 or −1, depending on their side. Thus, we could keep only those lines that have a slope above or below a certain threshold. For example, we could use a threshold value for the left lane markers so each line with a smaller slope could be a possible lane. For the right lanes, we shall use the positive equivalent and check if the slope is greater than it. This simple idea can reject more than 90% of the undesired lines.

Unfortunately, there are other objects on the side road that can be detected as lanes. The most common ones are parapets which, of course, we are not interested in. The rejection of those lines can be done based on a fundamental property of the road lanes. In the edge detection step, we can notice that a lane comprises two almost parallel and close lines. (See Figure 10). In the case of a double lane marker, there will be four such lines. Lane markers depend on the type of roads or even the country. Another property we can use to remove unwanted lines is the fact that lanes, in general, tend to be close to the center of the image, thus the lines that bound them will also be close to the center of the image. Using this property we can further remove unwanted lines by looking only at the lines that are closest to the center of the image.

The proposed method for finding real lane markers:

The first step is to search for each $\left(i, j\right)$ index pair in the $H$ accumulator, if there is any $H\left[i,j\right]$ value greater than a certain threshold. If so, the $\left(i,j\right)$ pair may be the $\left(\mathsf{\rho }, \mathsf{\theta }\right)$ values of the real lane.

Next, we should check for each line candidate previously detected if there are other possible candidates having a similar $\mathsf{\theta }$ value. We should be aware that even though the $\mathsf{\theta }$ values are close enough, the lines must be situated one near to the other so we should also check if the $\mathsf{\rho }$ value is almost the same.

In other words, for each highly voted candidate point $P$ in the accumulator $H$ the algorithm is looking for other high voted neighbors in a rectangle area centered in $P$. The rectangle’s width is two times the maximum $\mathsf{\theta }$ angle difference between the two lines whereas the length represents two times the maximum distance between the lane’s left and right edge. Figure 21 shows an example of four candidates, encompassed by green rectangles, which were found inside the red rectangle area defined by the candidate point.

Rectangle criteria:

If there is at least one other candidate line$Lj$for each candidate line$Li$in a rectangle centered in$Li$, then we can consider it a lane.

Any other candidate that does not meet the above criteria will be discarded. The algorithm must detect only one line for each lane marker, so if more candidates meet the criteria, we should consider only the most voted ones in each main bright area (Figure 11). A line average could also be used in this case.

Because of the quantization process, there may be some high-voted points in the accumulator that basically represent the same line (see Figure 21, where multiple bright pixels are next to each other). In this case, the criteria may select a high-voted line that is not an actual lane marker and find other candidates around it that represent the same line. Although a well-chosen threshold could reject the detection of fake lines, in this case, we can add another condition that states: if the two candidates are neighbors or too close to each other, they are considered as a single possible lane.

2.4. The Proposed Algorithm

To sum up the ideas presented in the last paragraphs we propose Algorithm 1:

Algorithm 1. The proposed lane detection algorithm.

Initialize parallelogram region of interest as undefined For each recorded frame Apply the Canny edge detector. If parallelogram region of interest is undefined: Define a trapezoidal region of interest for the road in front of the car based on the camera position. Detect lines using the probabilistic Hough transform. Detect two endpoints for each lane using the rectangle criteria. If both lanes were detected: Create a parallelogram region of interest for the two detected lines. Calculate the slope for each line. If slope is < 0: Classify line as left_lane Else Classify line as right_lane If left_lane slope > threshold_1 Indicate “Right turn” If right_lane slope < threshold_2 Indicate “Left turn” If !(left_lane slope > threshold_1) and !(right_lane slope < threshold_2) Indicate “Go forward” If left or right lane has not been detected Define a trapezoidal region of interest for the road in front of the car based on the camera position. Detect lines using the probabilistic Hough transform. Detect two endpoints for each lane using the rectangle criteria. If both lanes were detected: Create a parallelogram region of interest for the two detected lines. Calculate the slope for each line. If slope is < 0: Classify line as left_lane Else Classify line as right_lane If left_lane slope > threshold_1 Indicate “Right turn” If right_lane slope < threshold_2 Indicate “Left turn” If !(left_lane slope > threshold_1) and !(right_lane slope < threshold_2) Indicate “Go forward” Else Message “At least one lane has not been detected.” Set parallelogram region of interest as undefined.

3. Results

3.1. Number of Edge Pixels

One of the main purposes of the proposed method was to reduce the number of edge pixels using a parallelogram region of interest as lane position will not substantially change in the next frame.

The algorithm in Section 2.4 was able to remove a great number of unnecessary edge pixels that are not part of the actual lanes.

Table 1. Comparison between the number of edge pixels using a trapezoidal ROI and the number of edge pixels using a parallelogram ROI. (n = 40).

Frame	Trapezoidal ROI	Parallelogram ROI
1	4358	3394
2	4349	3420
3	4345	3419
4	7113	2491
5	5047	3187

shows a comparison between the trapezoidal ROI method and the proposed parallelogram ROI, where n, which represents the half-width of the parallelogram, was determined through trial and error to have a value of 40.

The frames used for testing were randomly chosen from a road drive recording [20].

Running the proposed algorithm for a total number of 969 frames resulted in a final average of 42% improvement in discarding unwanted edge pixels. Due to this optimization, we also obtain an increase in performance, our proposed method being approximately 30% faster, running in an average of 585.44 s as opposed to 850.39 s. Figure 22 shows the different runtimes of both methods, over 40 different runs on the entire dataset, in order to reduce the potential of certain runs being affected by outside factors and affecting our metrics. These tests were performed on an Intel i7-4710HQ processor.

The line detection algorithm using the probabilistic Hough transform will also suppress a random subset of edge pixels (see Section 2.3).

3.2. False Lanes Rejection

In Section 2.3 we presented a method for rejecting false lanes based on the propriety that each lane marker is composed of two close parallel edges.

Table 2. Comparison between the number of rejected false lanes using the two presented methods.

Frame	Threshold Value	Threshold Value and Rectangle Criteria	False Lanes in Frame
1	1	1	1
2	2	3	3
3	0	1	1
4	4	4	4
5	7	7	7
6	3	3	3
7	4	4	4
8	3	3	4
9	0	0	2

shows the number of false lanes rejected using only a threshold value and the number of false lanes rejected using the same threshold value and the rectangle criteria specified in Section 2.3.3. The rectangle used is 10 × 20 pixels, and the threshold is constant throughout the entire runtime

Moreover, using the same methods,

Table 3. Comparison between the number of rejected real lanes using the two presented methods.

Frame	Threshold Value	Threshold Value and Rectangle Criteria	Real Lane Markers in Frame
1	0	0	3
2	0	0	3
3	0	0	3
4	0	0	3
5	0	1	3
6	0	0	3
7	0	0	3
8	0	0	3
9	0	0	3

presents the number of real lanes that were rejected.

The results were achieved without using any slope restriction as presented in Section 2.3. The dynamic parallelogram region of interest method was not applied to the input image as it may itself remove false lanes.

The frames used for testing were selected from a road drive recording in which parapets, shadows, or other objects could generate undesired lines [20].

The algorithm has an 89% accuracy in rejecting false lane markers while the basic method rejected 86%.

In Table 2 we note that there is a small improvement in rejecting false lanes. However, in frame 5 the right lane was rejected because of the distance between the two-lane edges. (Table 3).

4. Discussion

Lane detection and tracking are great computer vision challenges, especially because the desired algorithm needs to be both fast and robust, to perform accurately in real-time. There are a lot of factors that can compromise the results and developing an algorithm suitable for each situation is not a handy task. The proposed algorithm highlights the significant number of observations that can lead to method optimization.

Most recent research tends to focus on optimizing both the speed at which the lanes are detected, as well as the accuracy of said detections, by various methods, which can be observed in [8,9,10,11,12,21]. In a similar fashion, our approach also focused on improving the results in [7], managing to increase both the computational efficiency, and the accuracy, by reducing the number of edge pixels that need to be computed. However, this method is not without its flaws, as we could observe from our results. Defining a wrong rectangle size may cause some real lane markers to be rejected, which is not desirable. This could be improved in future work by trying to automatically find fitting values to improve the robustness of the algorithm.

Something else to note is that the distance between the lane markers increases or decreases based on the overall position of the car, relative to the chosen window. This is something our current approach doesn’t address and could potentially be addressed in future work. A potential solution to this problem is to develop an adaptive algorithm that can adjust the dimensions of the rectangle based on the car’s position.

5. Conclusions

In this paper, we presented a method for performing lane detection by improving upon an existing one. By using a parallelogram ROI instead of a trapezoidal one, and applying the probabilistic Hough transform to detect the potential positions of the lines, we managed to obtain an increase in computational performance, as more than 40% of the initial edge pixels were removed, whilst also coming with an increase in accuracy of 3%.