Review on Lane Detection and Tracking Algorithms of Advanced Driver Assistance System

: Autonomous vehicles and advanced driver assistance systems are predicted to provide higher safety and reduce fuel and energy consumption and road trafﬁc emissions. Lane detection and tracking are the advanced key features of the advanced driver assistance system. Lane detection is the process of detecting white lines on the roads. Lane tracking is the process of assisting the vehicle to remain in the desired path, and it controls the motion model by using previously detected lane markers. There are limited studies in the literature that provide state-of-art ﬁndings in this area. This study reviews previous studies on lane detection and tracking algorithms by performing a comparative qualitative analysis of algorithms to identify gaps in knowledge. It also summarizes some of the key data sets used for testing algorithms and metrics used to evaluate the algorithms. It is found that complex road geometries such as clothoid roads are less investigated, with many studies focused on straight roads. The complexity of lane detection and tracking is compounded by the challenging weather conditions, vision (camera) quality, unclear line-markings and unpaved roads. Further, occlusion due to overtaking vehicles, high-speed and high illumination effects also pose a challenge. The majority of the studies have used custom based data sets for model testing. As this ﬁeld continues to grow, especially with the development of fully autonomous vehicles in the near future, it is expected that in future, more reliable and robust lane detection and tracking algorithms will be developed and tested with real-time data sets.


Introduction
Autonomous passenger vehicles are a direct implementation of transportation-related autonomous robotics research. They are also known as self-driving vehicles or driverless vehicles. Shakey the robot (1966)(1967)(1968)(1969)(1970)(1971)(1972) is the first autonomous mobile robot that has been documented [1]. It was developed by Stanford Research Institute's Artificial Intelligence Centre and was capable of detecting the environment, thinking, planning, and navigation. In basic settings, vision-based lane tracking and obstacle avoidance sparked interest in autonomous vehicles [2]. In the early 1990s, The Royal Armament Research and Development Establishment in the United Kingdom created two vehicles for obstacle-free navigation on and off the road [3]. In the United States, the first operations of autonomous driving in realistic settings dates back to Carnegie Mellon University's NavLab in the early 1990s [4]. The vehicle developed by NavLab was operated at very low speeds due to the limited computational power available at the time. Early US research projects also included the California PATH project, which developed the automated highway [5]. Vehicle steering was automated with manual longitudinal control in the "No Hands Across America" project [6]. In early 2000, CyberCars, one of several European projects began developing technologies based on automated transport [7]. The announcement of the defence advanced research projects agency (DARPA) grand challenge in 2003 generated research interest in Sustainability 2021, 13, 11417 2 of 29 autonomous cars. Following that, in 2006, the DARPA urban challenge was performed in a controlled situation with a variety of autonomous and human-operated vehicles. Since then, many manufactures, including Audi, BMW, Bosch, Ford, GM, Lexus, Mercedes, Nissan, Tesla, Volkswagen, Volvo and Google, have launched self-driving vehicle projects in collaboration with universities [8]. Google's self-driving car has experimented and travelled 500 thousand kilometres and has begun building prototypes of its own cars [9]. A completely autonomous vehicle would be expected to drive to a chosen location without any expectation of shared control with the driver, including safety-critical tasks.
The performance of lane detection and tracking depends on the well-developed roads and their lane markings, so smart cities are also a prominent factor in autonomous vehicle research. The idea of a smart city is often linked with an eco-city or a sustainable city, both of which seek to enhance the quality of municipal services while lowering their costs. Smart cities' primary goal is to balance technological innovation with the economic, social, and environmental problems that tomorrow's cities face. The greater closeness between government and people is required in smart cities that embrace the circular economy's concepts [10]. The way materials and goods flow around people and their demands will alter, as will the structure of cities. Several car manufacturers such as Tesla and Audi have already launched autonomous vehicle marketing for private use. Soon, society will be influenced by autonomous vehicles' spread to urban transport systems [11]. The development of smart cities with the introduction of connected and autonomous vehicles could potentially transform cities and guide long-term urban planning [10].
Autonomous vehicles and Advanced Driver Assistance Systems (ADAS) are predicted to provide a higher degree of safety and reduce fuel and energy consumption and road traffic emissions. ADAS is implemented for safe and efficient driving, which has many driver assistance features such as warning drivers about forwarding collision warning or safe lane change [12]. Research shows that most accidents occur because of driver errors, and the ADAS can reduce the accidents and workload of the driver. If there is a likelihood of an accident, ADAS can take the necessary action to avoid it [13]. Lane departure warning (LDW), which utilizes lane detection and tracking algorithms, is an essential feature of the ADAS. The LDW warns the driver when a vehicle crosses white lane lines unintentionally and controls the vehicle by bringing it back into the desired safe path. Three types of approaches for lane detection are usually discussed in the existing literature: learningbased approach, features-based approach, and model-based approach [13][14][15][16][17][18] (detailed analysis are presented in Section 3.2). Many challenges and issues have been highlighted in the literature regarding the LDW systems, such as visibility conditions change, variation in images, and lane appearance diversity [17]. Since different countries have used various lane markers, there is a challenge for lane detection and tracking to solve the problems.

Objectives and Scope of the Study
There are limited studies that provide state-of-art lane detection and tracking algorithms for ADAS. This review paper aims to comprehensively review the previous literature on lane detection and tracking for ADAS and identify gaps in knowledge for future research. The report compares different lane detection and tracking algorithms and analyses different datasets used to verify the algorithms and metrics used to evaluate the algorithms. Specifically, the review identifies and classifies the existing lane detection and tracking algorithms under three themes: features-based, learning-based and model-based, which provides a systematic approach towards understanding the key characteristics of lane detection and tracking algorithms in the literature. Some patented works by vehicle manufacturers under these three categories are also reviewed to acknowledge growing commercialisation interests in this field of study. However, given the large number of patents by educational institutions, research groups and vehicle manufacturers, a detailed review of patented works is outside the scope of this study. This systematic review is expected to assist researchers working in this area by delivering current advancements The structure of the paper is as follows. Section 2 provides an overview of the methodology adopted for the literature review. It is then followed by a detailed literature review that includes a brief introduction to sensors used in the ADAS, an analysis of the existing literature on lane detection and tracking algorithms. Section 4 presents the discussions followed by conclusions in Section 5.

Methodology
Literature was gathered from the electronic database. The database included "ISI Web of Science", "Science Direct" "Scopus" "Google Scholar" and "IEEE Xplore". The keywords used for the search were "Lane detection algorithms", "Lane tracking algorithms", "Lane departure warning algorithms", "Advanced driver assistance system", "Lane change tracking", "Vehicle tracking", Vehicle tracking sensors", and "Automated lane change" or a combination of these words ( Figure 1). We also searched for patented works. Patents published from 2006 to 2020 were also searched using the term "Lane detection and tracking," "Lane departure warning," and "Advance driver assistance system" using the "Google scholar" and "PubMed". As mentioned in Section 1.1, the objective of patents search is to acknowledge growing commercialisation interests in this field of study rather than a detailed review of the patented works. As such, only a sample of patents works from vehicle manufacturers was discussed. The period studied for the literature is the past two decades, as lane tracking and detection is an emerging field that has gained momentum post-2000. English language-based publications were only considered for the review as they are widely accessible to global readers. Relevant publications further improved the search procedure in the reference lists available in the collected literature. The lane detection and tracking algorithms were investigated under three approaches that have been commonly referred to in the literature (Features based, learning-based and model-based as shown in Figure 1). The existing databases were analyzed to identify the availability of datasets for future research. The lane detection criteria, calculation of the detection rate and accuracy of the algorithms that have been adopted in the literature are also reviewed. commercialisation interests in this field of study. However, given the large number of patents by educational institutions, research groups and vehicle manufacturers, a detailed review of patented works is outside the scope of this study. This systematic review is expected to assist researchers working in this area by delivering current advancements made in lane detection and tracking for ADAS and the challenges to overcome in the future for robust lane detection and tracking systems.
The structure of the paper is as follows. Section 2 provides an overview of the methodology adopted for the literature review. It is then followed by a detailed literature review that includes a brief introduction to sensors used in the ADAS, an analysis of the existing literature on lane detection and tracking algorithms. Section 4 presents the discussions followed by conclusions in Section 5.

Methodology
Literature was gathered from the electronic database. The database included "ISI Web of Science", "Science Direct" "Scopus" "Google Scholar" and "IEEE Xplore". The keywords used for the search were "Lane detection algorithms", "Lane tracking algorithms", "Lane departure warning algorithms", "Advanced driver assistance system", "Lane change tracking", "Vehicle tracking", Vehicle tracking sensors", and "Automated lane change" or a combination of these words ( Figure 1). We also searched for patented works. Patents published from 2006 to 2020 were also searched using the term "Lane detection and tracking," "Lane departure warning," and "Advance driver assistance system" using the "Google scholar" and "PubMed". As mentioned in Section 1.1, the objective of patents search is to acknowledge growing commercialisation interests in this field of study rather than a detailed review of the patented works. As such, only a sample of patents works from vehicle manufacturers was discussed. The period studied for the literature is the past two decades, as lane tracking and detection is an emerging field that has gained momentum post-2000. English language-based publications were only considered for the review as they are widely accessible to global readers. Relevant publications further improved the search procedure in the reference lists available in the collected literature. The lane detection and tracking algorithms were investigated under three approaches that have been commonly referred to in the literature (Features based, learningbased and model-based as shown in Figure 1). The existing databases were analyzed to identify the availability of datasets for future research. The lane detection criteria, calculation of the detection rate and accuracy of the algorithms that have been adopted in the literature are also reviewed.

Literature Review
A comparison of the different sensors used in ADAS is presented first. It is then followed up with an in-depth review of algorithms used for lane detection and tracking, including the patented works.

Sensors Used in the ADAS
ADAS deploy different sensors fusion systems to guide the vehicle ( Figure 2). In the literature, three types of sensors have been identified that are used in the ADAS, which are, Light Amplification by Stimulated Emission of Radiation (LASER) based sensors, Radio Detection And Ranging (RADAR) based sensors and vision-based sensors as described below. Table 1 shows the Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis of LASER, RADAR and vision-based sensors.se Table 1. SWOT analysis of sensors that are used in ADAS. Laser scanner and Light detection and ranging (LIDAR) is the common laser-based sensors. In this technology, the transmitter and receiver are placed, and the impulse light of electromagnetic waves are recorded through it. Infrared near about (800-950 nm) and ultraviolet above (1500 nm) wavelength of the electromagnetic spectrum are used [19]. By estimating the time of flight, the distance between the transmitter and the receiver is calculated. It may not be possible to derive the direct relative velocity of the moving object, so it is obtained by taking the derivative of ranges with respect to time. These types of sensors are used for multiple target tracking. The drawbacks of these sensors are vulnerability to dirty lenses and the inadequacy of the reflecting target. Besides, for weather conditions, these sensors may be too sensitive. These types of sensors are reliable for automatic car parking and collision mitigation [19,20].

RADAR Based Sensors
RADAR sensors can detect the images in haze, dust, rain, and snow up to 200 m ahead. Through the radar detection and ranging process, these sensors emit strong radio waves through the transmitter and receive them back through the receiver, similar to laser-based sensors. The distance between sensors and objects is calculated by the time of flight. Another advantage is that frequency between emitted and Doppler echo can be calculated, which provides the object's velocity. To map movements of aircraft, these kinds of sensors are often used in aviation and defence manufacturing sectors. In the automobile sector, two types of sensors are used: long-range sensors, which range between 77-81 GHz spectrum and short-range sensors that range between 21. 65-26.65 GHz. In extreme weather conditions, these sensors are very vulnerable and sometimes fail to work. These kinds of sensors are used for collision mitigation and adaptive cruise control [19,20].

Vision-Based Sensors
These types of sensors come under the passive sensors category, which means they do not emit any waves. To assess the presence, orientation and accuracy of the surrounding objects, vision sensors use images. Vision sensors use a mixture of image acquisition and image processing, and multi-point inspection is carried out using a single sensor. Two types of sensors are used in a vision-based system, the first is a monocular camera, and the second is a stereo vision camera. These sensors do not directly derive the range and velocity of the objects, and as such, a sophisticated signal procedure is used to derive these parameters. These sensors are readily available and affordable in the automobile sector. For traffic signal analysis and lane change assistance, these kinds of sensors are applicable. The main drawback is vulnerabilty to extreme weather conditions and sometimes failing to work at nighttime [19,20]. 77-81 GHz spectrum and short-range sensors that range between 21. 65-26.65 GHz. In extreme weather conditions, these sensors are very vulnerable and sometimes fail to work. These kinds of sensors are used for collision mitigation and adaptive cruise control [19,20].

Vision-Based Sensors
These types of sensors come under the passive sensors category, which means they do not emit any waves. To assess the presence, orientation and accuracy of the surrounding objects, vision sensors use images. Vision sensors use a mixture of image acquisition and image processing, and multi-point inspection is carried out using a single sensor. Two types of sensors are used in a vision-based system, the first is a monocular camera, and the second is a stereo vision camera. These sensors do not directly derive the range and velocity of the objects, and as such, a sophisticated signal procedure is used to derive these parameters. These sensors are readily available and affordable in the automobile sector. For traffic signal analysis and lane change assistance, these kinds of sensors are applicable. The main drawback is vulnerabilty to extreme weather conditions and sometimes failing to work at nighttime [19,20].

Lane Detection and Tracking Algorithms
In this section, we have conducted a comparison and analysis of algorithms in three different categories according to approaches used: features-based approach, model-based and learning-based approach.
The feature-based approach uses edges and local visual characteristics of interest, such as gradient, colour, brightness, texture, orientation, and variations, which are relatively insensitive to road shapes but sensitive to illumination effects. The model-based approaches apply global road models to fit low levels of features that are more robust against illumination effects, but they are sensitive to road shapes [13,14]. The geometrics parameters are used in the model-based approach for lane detection [16][17][18]. The learningbased approach consists of two stages: training and classification. The training process uses previously known errors and system properties to construct a model, e.g., program variables. In addition, the classification phase applies the training model to the user set of properties and outputs that are more likely to be correlated with the error ordered by their

Lane Detection and Tracking Algorithms
In this section, we have conducted a comparison and analysis of algorithms in three different categories according to approaches used: features-based approach, model-based and learning-based approach.
The feature-based approach uses edges and local visual characteristics of interest, such as gradient, colour, brightness, texture, orientation, and variations, which are relatively insensitive to road shapes but sensitive to illumination effects. The model-based approaches apply global road models to fit low levels of features that are more robust against illumination effects, but they are sensitive to road shapes [13,14]. The geometrics parameters are used in the model-based approach for lane detection [16][17][18]. The learning-based approach consists of two stages: training and classification. The training process uses previously known errors and system properties to construct a model, e.g., program variables. In addition, the classification phase applies the training model to the user set of properties and outputs that are more likely to be correlated with the error ordered by their probability of fault discloser [19]. In the following sections, we describe the three approaches used in the literature in detail. It is then followed up by summary tables (Tables 2-5) that present the key features of these algorithms and strengths, weaknesses, and future prospects.

Features-Based Approach (Image and Sensor-Based Lane Detection and Tracking)
Image and sensor-based lane detection and tracking decision-making processes are dependent on the sensors attached to the vehicle and the camera output. In this approach, the image frames are pre-processed, and a lane detection algorithm is applied to determine lane tracking. The sensor values are used to further decide on the path to be followed by the lane markings [22,23].
Kuo et al. [24] implemented a vision-based lane-keeping system. The proposed system obtains the vehicle position following the lane and controls the vehicle to be in the desired path. The steps involved in the lane-keeping system are inverse perspective mapping, detection of lane scope features and reconstruction of the lane markings. The main drawback of the system is that the performance is reduced when the vehicle is driving in a tunnel.
Kang et al. [25] proposed a kinematic-based fault-tolerant mechanism to detect the lane even if the camera cannot provide the road image due to malfunction or environmental constraints. In the absence of camera input, the lane is predicted using the kinematic model by taking the parameters such as the length and speed of the vehicle. The camera input is given as a clothoid cubic polynomial curve road model. In the absence of camera input, the lane coefficients of the clothoid model will be available. A lane restoration scheme is used to overcome this loss based on a multi-rate state estimator obtained from the kinematic lateral motion model in the clothoid function. The predicted lane is based on the past curvature rate and road curvature. The results show that the proposed method can maintain the lane for 3 s without camera input. The developed algorithm was simulated using CARSIM and Simulink. It has been tested in a test vehicle equipped with an Auto Box from dSPACE in Tucson from HYUNDAI Motors.
Borkar et al. [26] proposed a lane detection and tracking method using inverse projective mapping (IPM) to create a bird's-eye view of the road; a Hough transform for detecting candidate lane and Kalman filter track the lane. The road image is converted to grayscale form followed by temporal blurring. The application of IPM makes the image provide a bird's eye view. The lanes are detected by identifying the pair of parallel lines which are separated by a distance. The IPM images are converted to binary, and a Hough transform is performed on the binary image and then divided into two halves. To determine the center of the line, the one-dimensional matched filter is applied to each sample. The pixel with a large correlation that exceeds the threshold is selected as the center of the lane. The Kalman filter is used to track the lane, which takes the lane orientation and difference between the current and previous frames. A firewire camera is used to capture the image of the road. The performance of the proposed algorithm provides better accuracy under the isolated highway and metro highway, and the accuracy is in the range of 86% on city roads. The improved performance is due to the usage of the Kalman filter to track the lane.
Sun et al. [27] proposed a lane detection mechanism considering multiple frames in contrast with the single frame along with the inertial work classifier. The initially assigned probability value changes due to error and vehicle movement. Kalman filter is applied to smooth the line segments in Hough space. The inertial measurement unit (IMU) values are used to align the previous line segments in the Hough space. The lane detection is determined by considering the line segments with a high probability value. The analysis of the method using the Caltech dataset provides accuracy in the range of 95% to 97%. The lane detection under different environmental conditions such as sunlight, rain and with high values of sunlight and rainfall shows the performance in the range of 72% to 87%. The Hough transform is employed to extract the line segment from lane markings stored in the Hough space. The Hough space is used to store the line segments with an associated probability value. The truthiness of the line segments is determined using Convolutional Neural Net. The system is implemented using NVIDIA GTX1050ti GPU, OV10650 camera, and the IMU is Epson G320.
Lu et al. [28] proposed a lane detection algorithm for urban traffic scenarios in which the road is well-constructed, flat and of equal width. The road model is constructed using feature line pairs (FLP), the FLP is detected using Kalman filter and a regression diagnostic technique to determine the road model using FLP. The result shows that the time taken to detect the road parameters is 11 ms. The proposed method is implemented using C++ on a 1.33 GHz AMD processor-based personal computer with a single camera and a Matrox Meteor RGB/PPB digitizer and implemented in THMR-V (Tsinghua Mobile Robot V).
Zhang and Shi [29] proposed a lane detection method for detecting the lanes at night. The sober and canny operator detects the edges of the lanes. Gradients acquiring a certain threshold are labelled as edge points. The histogram with the higher brightness is named as lane boundary, and the low valued histogram is named a road. The accuracy of the proposed method is high even in the presence of noises from car head and rear lights and road contour signs.
Borkar et al. [30] proposed a layered approach to detect the lane at night. The region of interest is specified in the captured image of the road. The image is converted to greyscale for further processing. Temporal burring is applied to obtain the continuous lanes of the long line. Depending on the characteristics of the neighboring pixels, an adaptive controller is used to determine the object. The images are converted to the left and right halves, and each half Hough transform is performed to determine the straight lines. The final process deals with the fitting of all the straight lines. Firewire S400 (400 Mbps) color camera in VGA resolution (640 × 480) at 30 fps is used to capture the video and fed to MATLAB, and lanes are detected in an offline manner. The performance of the proposed method is good in isolated highways and in metro highway scenarios. With moderate traffic, the accuracy of detecting the lanes is reduced to 80 percent.
Priyadarshini et al. [31] proposed a lane detection system that detects the lane during the daytime. The captured video is converted to a grayscale image. A Gaussian filter is applied to remove the noise. The Canny edge detection algorithm is used to detect the edges. To identify the length of the lane, a Hough transform is applied. The proposed method is simulated using a raspberry pi-based robot with a camera and ultrasonic sensors to determine the distance between neighbouring vehicles.
The survey by Hong et al. [32] discussed video processing techniques to determine the lanes illumination change on the region of interest for straight-line roads. The survey highlights the methodologies involved, such as choosing the proper color space and determination of the region of interest. Once the intended image is captured, a color segmentation operation is performed using region splitting and clustering schemes. This is followed by applying the merging algorithm to suppress the noise in the image.
A color-based lane detection and a representative line extraction algorithm are proposed by Park et al. [33]. The captured image in RGB format is converted to gray code followed by binary image conversion. The purpose of binary image conversion is to remove the shadows in the captured image. The lanes in the image are detected using the canny algorithm by the feature named color. The direction and intensity are determined by removing the noise using the gaussian filter. The images are smoothened by applying a median filter. The lanes in the image are considered as the region of interest, and Hough transform is applied to confirm the accuracy of the lanes in the region of interest. The experiment is performed during the daytime. The results show that the lane detection rate is more than 93%.
El Hajjouji et al. [34] proposed a hardware architecture for detecting straight lane lines using Hough transform. The CORDIC (Coordinate Rotation Digital Computer) algorithm calculates the gradient and phase from the captured image. The output of CORDIC block is the norm and angle of the x-axis of the image. The norm and angles are compared with the threshold obtained from the region of interest. The Hough transform is applied to the outcome of the comparator module, and the relation between the Hough space and the angle is determined. The noises are removed by the Hough transform voting procedure. Finally, the output is obtained as the slope of the straight line. The algorithm is implemented in the Virtex-5 ML505 platform. The algorithm was tested on a variety of images with varying illumination and different road conditions, such as urban streets, highways, occlusion, poor line paintings, day and night and scenarios. The algorithm provides a detection rate of 92%.
Samadzadegan et al. [35] proposed a lane detection methodology in a circular arc or parabolic based geometric method. The RGB colour is converted to an intensity image that contains a specific range of values. A three-layer pyramid image is constructed using bi-cubic interpolation method. Among the three layers of region of interest, the first layer pixels undergo randomized Hough transformation to determine the curvature and orientation features followed by a Genetic Algorithm Optimisation. The process is repeated to the remaining two layers. The outcome obtained in the lower layers are the features of the lane and used to determine the lanes in the region of interest. The result shows that there is a performance drop in lane detection when entering the tunnel region and occlusion in lane markings due to the shadow of another vehicle.
Cheng et al. [36] proposed a hierarchical lane detection system to detect the lanes on structured and unstructured roads. The system classifies the environment into structured and unstructured based on the feature extraction, which depends on the color of the lane marking. The connected component labelling method is applied to determine the feature objects. During the training, phase supervised learning is performed and manually classified the objects as left lane, right lane and no lane markings. The image is classified as structured and unstructured based on the vote value associated with the weights. The lanes for structured roads are detected by eliminating the moving vehicle on the lane image followed by lane recognition by considering the angle of inclination and starting points of the lane markings. The lane coherence verification module compares the lane width of the current frame with the previous frame to determine the lanes. For unstructured roads, the following steps are performed: mean shift segmentation, which deals with the determination of road surface by comparing with the surroundings to determine the variation in colors and texture. The region merging and boundary smoothing module deals with pruning unnecessary boundary lines and neglecting the region which is smaller than the threshold. The boundary is selected based on the posterior probability of each set of candidates. The simulation results show that around 0.11 s is needed to identify structured or unstructured roads. The system achieves an accuracy of 97% in lane detection.
Han et al. [37] proposed a LIDAR sensor-based road boundary detection and tracking for both structured and unstructured roads. The LIDAR is used to obtain the polar coordinates. The line segments are obtained from the height and pitch of LIDAR. Information such as roadside, curbs, sidewalks and buildings are obtained from the line segments. The road slope and width are obtained by merging two-line segments. The road is tracked using the nearest neighbor filter to estimate the state of the target. The algorithm is tested in a real vehicle equipped with LIDAR, GPS and IMU. The road boundary detection accuracy is 95% for structured and 92% for unstructured roads.
Le et al. [38] proposed a method to detect pedestrian lanes under different illumination conditions with no lane markings. The first stage of the proposed system is the vanishing point estimation which works based on votes of local orientations from colored edge pixels. The local orientation of pixels is determined as the vanishing point. The next stage is the determination of the sample region of the lane from the vanishing point. To achieve higher robustness towards different illuminations, invariant space is used. Finally, the lanes are detected using the appearance and shape information from the input image. A Greedy algorithm is applied, which helps to determine the connectivity between the lanes in each iteration of the input image. The proposed model is tested on the input image of both indoor and outdoor environments. The results show that the lane detection accuracy is 95%.
Wang et al. [39] proposed a lane detection system for straight and curve road scenarios. The captured image determines the region of interest, set as 60 m which falls in the near field region. The region of interest is divided into the straight region and the curve region. The near field region is approximated as the straight line, and the far-field region is approximated as the curve. An improved Hough transform is applied to detect the straight line. The curve is determined in the far-field region using the least-squares curve fitting method. The WAT902H2 camera model is used to capture the image of the road. The results show that the time taken to determine the straight and curve lane is 60-80 ms compared to 70-100 ms in the existing works and the accuracy is around 92-93%. The error rate in bending to the left or right direction is from −0.85 to 5.20% for different angles.
Yenıaydin [40] proposed a lane detection algorithm based on camera and 2D LIDAR input data. The camera obtains the bird's eye view of the road, and the LIDAR detects the location of objects. The proposed method consists of the steps mentioned below:

•
Obtain the camera and 2D LIDAR data. • Perform segmentation operation of the LIDAR data to determine groups of objects. It is done based on the distance among different points.

•
Map the group or objects to the camera data.

Model-Based Approach (Robust Lane Detection and Tracking)
Lee and Moon [42] proposed a robust lane detection and tracking system. This system's main aim is to detect the lane and track by considering different environmental conditions such as clear sky, rainy, and snowy during morning and night. The proposed system consists of three phases, namely initialization, lane detection, and lane tracking. In the initialization phase, the road region is captured and pre-processed to a low-resolution image. The edges are extracted, and the image is split into the left half and right half region. An intersection point is made from both regions, and intersection points are mostly found near the vanishing point. Once the vanishing points become greater than the threshold, the region above and below the vanishing points is removed. In the lane marking detection phase, the lane marking is determined in the rectangular region of interest. The image is converted into greyscale by using edge line detection, and a line segment is detected. The hierarchical agglomerative clustering method is used for a color image. The line segment is determined from surrounding vehicles, shadows, trees, and buildings by using its frequency in the region of interest. Other disturbances are not continuous compared to the real lane marking, and they can be determined by comparing them with the consecutive frames. In the lane tracking phase, lane tracking is achieved from the modified region of interest. Multiple pairs of lanes with the same weight are considered, and the smallest are chosen. Some lanes, which are not detected, are predicted by using the Kalman filter. This system is tested using C++ and open CV library with Ubuntu14. There is scope for improvement of the algorithm during the night scenario.
Son et al. Chae et al. [46] proposed an autonomous lane changing system consisting of three modules: perception, motion planning, and control. The surrounding vehicles are detected using LIDAR sensor input. In motion planning, the vehicle determines the mode such as lane-keeping or lane change, followed by the desired motion that is planned considering the safety of surrounding vehicles. A linear quadratic regulator (LQR) based model predictive control is used for longitudinal acceleration and deciding the steering angle. The stochastic model predictive control is used for lateral acceleration.
Chen et al. [47] proposed a deep convolutional neural network to detect the lane markings. The modules involved in the lane detection process are lane marking generation, grouping, and lane model fitting. The lane grouping process involves forming a cluster comprising neighbouring pixels represented as a single label that belongs to the same lane and connecting the labels called super marking. The next step of lane model fitting uses 3rd order polynomial to represent straight and curved lanes. The simulation is done on the CAMVID dataset. The setup requires high-end systems to do the training. The algorithm is evaluated for a minimal real-time situation. The authors proposed a Global Navigation Satellite System (GNSS) based lane-keeping assistance system, which calculates the target steering angle using a model predictive controller. The advantage of the approach is that it is estimated from GNSS when the lane is not visible due to environmental constraints. The steering angle and acceleration are modelled using the first-order lag system. The model predictive control is used to control the lateral movement of the vehicle. The proposed system was simulated, and prototype testing was conducted in a real vehicle, OUTLANDER PHEV (Mitsubishi Motors Corporation). The results show that the lane is followed with a minimal lateral error of about 0.19 m. The drawback of the approach is that the time delay of GNSS has an impact on the oscillation in the steering. Hence, the GNSS time delay should be kept minimal compared to the steering time delay.
Lu et al. [48] proposed a lane detection approach using Gaussian distribution random sample consensus (G-RANSAC). The process involves converting a bird's eye view image to look at all the lane characteristics. The next step is using a ride detector to extract the features of lane points and remove noise points using an adaptable neutral network. The ridge features are extracted from the gray images, which provide better results during the presence of vehicle shadow and minimal illumination on the environment. Finally, the lanes are detected using the RANSAC approach. The RANSAC algorithm considers the confidence level of ridge points in determining the lanes from noise. The proposed algorithm is tested under four different illumination conditions: normal illumination and good pavement, intense illumination and shadow interruption, normal illumination and sign-on-the-ground interruption and poor illumination and vehicle interference. The algorithm achieved 99.02%, 96.92%, 96.65% and 91.61% true-positive rates respectively.

Learning-Based Approach (Predictive Controller Lane Detection and Tracking)
Bian et al. [49] implemented a lane-keeping assistance system (LKAS) with two switchable assistance modes: lane departure prevention and lane-keeping co-pilot modes. The LKAS is designed to achieve better reliability. The two switchable assistance modes consist of a conventional Lane Departure Prevention (LDP) mode and a lane-keeping Co-pilot (LK Co-Pilot) mode. The LDP mode is activated if a lane departure is detected. A lateral offset is used as a lane-departure metric to determine whether to trigger the LDP or not. The LK Co-pilot mode is activated if the driver does not intend to change the lane; this mode helps the driver follow the expected trajectory based on the driver's dynamic steering input. Care should be taken to set the threshold accurately and adequately; otherwise false lane detection would be increased.
Wang et al. [50] proposed a lane-changing strategy for autonomous vehicles using deep reinforcement learning. The parameters which are considered for the reward are delay and traffic on the road. The decision to switch lanes depends on improving the reward by interacting with the environment. The proposed approach is tested under accident and non-accident scenarios. The advantage of this approach is collaborative decision making in lane changing. Fixed rules may not be suitable for heterogeneous environmental or traffic scenarios.
Wang et al. [51] proposed a reinforcement learning-based lane change controller for a lane change. Two types of lane change controllers are adopted, namely longitudinal and lateral control. A car-following model, namely the intelligent driver model, is chosen for the longitudinal controller. The lateral controller is implemented by reinforcement learning. The reward function is based on yaw rate, acceleration, and time to change the lane. To overcome the static rules, a Q-function approximator is proposed to achieve continuous action space. The proposed system is tested in a custom-made simulation environment. Extensive simulation is expected to test the efficiency of the approximator function under different real-time scenarios.
Suh et al. [52] implemented a real-time probabilistic and deterministic lane changing motion prediction system which works under complex driving scenarios. They designed and tested the proposed system on both a simulation and real-time basis. A hyperbolic tangent path is chosen for the lane-change maneuver. The lane changing process is initiated if the clearance distance is greater than the minimum safe distance and the position of other vehicles. A safe driving envelope constraint is maintained to check the availability of nearby vehicles in different directions. A stochastic model predictive controller is used to calculate the steering angle and acceleration from the disturbances. The disturbance values are obtained from experimental data. The usage of advanced machine learning algorithms could improve the currently developed system's reliability and performance.
Gopalan et al. [53] proposed a lane detection system to detect the lane accurately under different conditions such as lack of prior knowledge of the road geometry, lane appearance variation due to change in environmental condition, and independent of vehicle speed. The modules of the proposed system are lane detection and tracking. The basic approach used for lane detection is to classify the lane markings from the non-lane markings from the labelled training sample. A pixel hierarchy feature descriptor method is proposed to identify the correlation between the lane and its surroundings. A machine learning-based boosting algorithm is used to identify the most relevant features. The advantage of the boosting algorithm is the adaptive way of increasing or decreasing the weightage of the samples. The lane tracking process is performed during the non-availability of knowledge about the motion pattern of lane markings. Lane tracking is achieved by using particle filters to track each of the lane markings and understand the cause for the variation. The variance is calculated for different parameters such as the initial position of the lane, motion of the vehicle, change in road geometry, traffic pattern. The variance associated with the above parameters is used to track the lane under different environmental conditions. The learning-based proposed system provides better performance under different scenarios. The point to consider is that the assumption made is the flat nature of the road. The flat road image was chosen to avoid the sudden appearance and disappearance of the lane. The proposed system is implemented at the simulation level.
To summarize the progress made in lane detection and tracking as discussed in this section, Table 2 has been presented that shows the key steps involved in the three approaches for lane detection and tracking, along with remarks on their general characteristics. It is then followed with Tables 3-5 that presents the summary of data used, strengths, drawbacks, key findings and future prospects of the key studies that have adopted the three approaches in the literature.    Hough transform improves the output of the lane tracker.
--Performance of the proposed system is better.
Raspberry pi based robust with camera and sensors.
Simulation of the proposed method by using raspberry Pi based robot with a monocular camera and radar-based sensors to determine the distance between neighboring vehicles. Better accuracy in the day time.
Algorithm needs changes to test in different scenario.
The results show that the lane detection rate is more than 93%.

MATLAB
There is scope to test the algorithm in the night time.

Custom data
Unwanted noise reduces the performance of the algorithm.
[    Provides better results during the presence of vehicle shadow and minimal illumination of the environment.

--
The proposed algorithm is tested under different illumination condition ranging from normal, intense, normal and poor and provides lane detection accuracy as 95%, 92%, 91% and 90%.

Software based analysis
Need to test proposed method under various times like day, night.
Test vehicle --  Based on the review, some of the key observations from Tables 3-5 are summarized  below: • Frequent calibration is required for accurate decision making in a complex environment.

•
Reinforcement learning with the model predictive control could be a better choice to avoid false lane detection.

•
Model-based approaches (robust lane detection and tracking) provide better results in different environmental conditions. Camera quality plays an important role in determining lane marking.

•
The algorithm's performance depends on the type of filter used, and the Kalman filter is mostly used for lane tracking.

•
In a vision-based system, image smoothing is the initial lane detection and tracking stage, which plays a vital role in increasing systems performance. • External disturbances like weather conditions, vision quality, shadow and blazing, and internal disturbances such as too narrow, too wide, and unclear lane marking, drop algorithm performance.

•
The majority of researchers (>90%) have used custom datasets for research. • Monocular, stereo and infrared cameras have been used to capture images and videos. The algorithm's accuracy depends on the type of camera used, and a stereo camera gives better performance than a monocular camera.

•
The lane markers can be occluded by a nearby vehicle while doing overtake.

•
There is an abrupt change in illumination as the vehicle gets out of a tunnel. Sudden changes in illumination affect the image quality and drop the system performance.

•
The results show that the lane detection and tracking efficiency rate under dry and light rain conditions is near 99% in most scenarios. However, the efficiency of lane marking detection is significantly affected by heavy rain conditions. • It has been seen that the performance of the system drops due to unclear and degraded lane markings. • IMU (Inertia measurement unit) and GPS are examples that help to improve RADAR and LIDAR's performance of distance measurement.

•
One of the biggest problems with today's ADAS is that changes in environmental and weather conditions have a major effect on the system's performance.

Patented Works
According to the patent's family size, it is observed that Toyota has a generally greater number of patents work (521), followed by Ford (406), General Motors (GM) (353), Honda motor (284) and Uber (245). Six of the top ten companies are from the United States, while four are from Asia. From a patent standpoint, Europe seems to be lagging behind in the battle for ADAS, and that the patents published in China and other Asian countries for lane detection and tracking are invented in the universities. Only Google and General Motor patent portfolios have a high technical relevance score among the top ten patent manufacturers. On the other hand, all portfolios have an above average market coverage score, indicating that their manufacturer believes their inventions are valuable enough to protect globally, and it highlights the significance and promises that companies perceive in autonomous driving. The detailed review of the patent works is beyond the scope of this study. However, given the commercial nature of lane detection and tracking, a sample of patented works, especially from the vehicle manufacturer, that align with the three approaches (feature-based, learning-based and model-based) has been presented in Table 6. Some of the key observations from Table 6  The positive and negative ratios of the extracted characteristics of the frame are used to assess the system's correctness. The degree of accuracy is enhanced by including the judgment in all extracted frames [58]. • At a present calculation cycle, the lane change assistance calculates a target control amount comprising a feed-forward control using a target curvature of a track for changing the host vehicle's lane [59]. • Extra details analyzing signals mounted to determine if a collision between the host vehicle and any other vehicle is likely to occur, allowing action to be done to avoid the accident [60].

•
There are two kinds of issues that are often seen and corrected in dewarped perspective images: a stretching effect at the periphery region of a wide-angle image de warped by rectilinear projection, and duplicate images of objects in an area where the left and right camera views overlap [61]. • The object identification system examines the pixels in order to identify the object that has not previously been identified in the 3D Environment [62].

Discussion
Based on the review of studies on lane detection and tracking in Section 3.2, it can be observed that there are limited data sets in the literature that researchers have used to test lane detection and tracking algorithms. Based on the literature review, a summary of the key data sets used in the literature or available to the researchers is presented in Table 7, which shows some of the key features, strengths, and weaknesses. It is expected that in future, more data sets may be available for the researchers as this field continues to grow, especially with the development of fully autonomous vehicles. As per the statistics survey of research papers published between 2000 and 2020, almost 42% of researchers mainly focused on Intrusion Detection System (IDS) matrix to evaluate the performance of the algorithms. This may be because the efficiency and effectiveness of IDS are better when compared to Point Clustering Comparison, Gaussian Distribution, Spatial Distribution and Key Points Estimation methods. The verification of the performance of the algorithms for lane detection and tracking system is done based on ground truth data set. There are four possibilities as true positive (TP), false negative (FN), false positive (FP) and true negative (TN), as shown in Table 8. There are many metrics available for the evaluation of performance, but the most common are accuracy, precision, F-score, Dice similarity coefficient (DSC) and receiver operating characteristic (ROC) curves. Table 9 provides the common metrics and the associated formulas used for the evaluation of the algorithms. For unseen or occluded lane marking annotated manually with a cubic spline.
Except for four lanes markings, others are not annotated Caltech [64] 10 h video 640 × 480 Hz of regular traffic in an urban environment. 250,000 frames, 350,000 boundary boxes annotated with occlusion and temporal.
Entire dataset annotated, testing data also provided (set 06-set 10) and training data (set 00-set 05) each 1 GB.
Not applicable for all types of road geometries and weather conditions. Lane detection challenge, velocity estimation challenge and ground truths have been provided.
Calibration file for lane detection has not been provided.
UAH [68] Raw real time data: Raw-GPS, RAW-Accelerometers. Processed data as continuous variables: pro lane detection, pro vehicle detection and pro OpenStreetMap data. Processed data as events: events list lane changes and events inertial.
More than 500 min naturistic driving and processed sematic information have provided. Limited accessibility to the research community BDD100K [69] 100,000 videos for more than 1000 h, road object detection, drivable area, segmentation and full frame sematic segmentation.
IMU data, timestamp and localization have been included in the dataset. Data for unstructured road has not covered.  If the database is balanced, the accuracy rate should accurately reflect the algorithm's global output. The precision reflects the goodness of optimistic forecasts. The greater the accuracy, the lower the number of "false alarms." The recall, also called true positive rate (TPR), is the ratio of positive instances that are correctly detected by the algorithm. Therefore, the higher the recall, the higher the algorithm's quality in detecting positive instances. The F1-Score is the Precision and Recall harmonic mean, and since they are combined into a concise metric, it can be used for comparing algorithms. Because it is more sensitive to low values, the harmonic mean is used rather than arithmetic. Hence, a valid algorithm has a satisfactory F1 score if it has accuracy and high recall. These parameters can be estimated as unique metrics for each class or as the algorithm's overall metrics [73]. Table 10 shows the SWOT analysis of different approaches used for lane detection and tracking algorithms. The use of a Learning-based approach (model predictive controller) is considered an emerging approach for lane detection and tracking because it is computationally more efficient than the other two approaches, and it provides reasonable results in real-time scenarios. However, the risk of mismatching lanes and performance drop in inclement weather conditions are the drawback of the learning-based approach. Featurebased approach, while time-consuming, can provide better performance in optimization of lane detection and tracking. However, this approach poses challenges in handling high illumination or shadows. Image and sensor-based lane detection and tracking approaches have been used widely in lane detection and tracking patents. In addition, from the literature synthesis, several gaps in knowledge are identified and are presented in Table 11. The literature review shows that clothoid and hyperbola shape roads are ignored for lane detection and algorithms road because of the complexity of road structure and unavailability of the dataset. Likewise, much work has already been done on structured roads' pavement marking compared to unstructured roads ( Figure 3). Most studies focus on straight roads. It is to be noted that unstructured roads are available in residential areas, hilly area roads, forest area roads. Much research has previously considered daytime, while night and rainy conditions are less studied. From the literature, it is observed that, in terms of speed flow conditions, they have been previously researched on the speed levels of 40 km/h to 80 km/h while high speed (above 80 km/hr) has received less attention. Further, occlusion due to overtaking vehicles or other objects (Figure 4), and high illumination also pose a challenge for lane detection and tracking. These issues should be addressed to move from level 3 automation (partial driving) to level 5 fully autonomous Also, new databases for more testing of algorithms are needed as researchers are constrained due to the unavailability of datasets. There is, however, the prospect of using synthetic sensor data generated by using a test vehicle or driving scenario designing through a driving simulator app available through commercial software.        Lane markings are usually yellow and white, although reflector lanes are designated with other colors. The number of lanes and their width varies per country. Due to the existence of shadows, there may be problems with vision clarity. The surrounding cars may obstruct the lane markings. Likewise, there is a dramatic shift in lighting as the car exits a tunnel. As a result, excessive light has an impact on visual clarity. Due to different weather conditions such as rain, fog, and snow, the visibility of the lane markings decreases. In the evening, visibility may be reduced. These difficulties in lane recognition and tracking lead to a drop in the performance of lane detection and tracking algorithms. Therefore, the development of a reliable lane detecting system is a challenge.

Conclusions
Over the last decade, many researchers have researched ADAS. This field continues to grow, as fully autonomous vehicles are predicted to enter the market soon [80,81]. There are limited studies in the literature that provides the state-of-art in lane detection and tracking algorithms and evaluation of the algorithms. To fulfil this gap, in this study, we have provided a comprehensive review of different methods of lane detection and tracking algorithms. In addition, we presented a summary of different data sets that researchers have used to test the algorithms, along with the approaches for evaluating the performance of the algorithms. Further, a summary of patented works has also been provided.
The use of a Learning-based approach is gaining popularity because it is computationally more efficient and provides reasonable results in real-time scenarios. The unavailability of rigorous and varied datasets to test the algorithms have been a constraint to the researchers. However, using synthetic sensor data generated by using a test vehicle or driving scenario through a vehicle simulator app availability in commercial software has opened the door for testing algorithms. Likewise, the following areas need more investigations in future: This study aimed to comprehensively review previous literature on lane detection and tracking for ADAS and identify gaps in knowledge for future research. This is important because limited studies provide state-of-art lane detection and tracking algorithms for ADAS and a holistic overview of works in this area. The quantitative assessment of mathematical models and parameters is beyond the scope of this work. It is anticipated that this review paper will be a valuable resource for the researchers intending to develop reliable lane detection and tracking algorithms for emerging autonomous vehicles in future.
Author Contributions: Investigation, data collection, methodology, writing-original draft preparation, S.W.; Supervision, writing-review and editing, N.S.; Supervision, writing-review and editing, P.S. All authors have read and agreed to the published version of the manuscript.