Adaptive Intervention Algorithms for Advanced Driver Assistance Systems

Abstract

Advanced driver assistance systems (ADASs) have recently gained popularity as they assist vehicle operators in staying within safe boundaries, helping them thereby to prevent possible collisions. However, despite their success and development, most ADAS use common and deterministic warning thresholds for all drivers in all driving environments. This may occasionally lead to the issuance of annoying inadequate warnings, due to the possible differences between drivers, the changing environments and driver statuses, thus reducing their acceptance and effectiveness. To fill this gap, this paper proposes adaptive algorithms for commonly used warnings based on real-time traffic environments and driver status including distraction and fatigue. We proposed adaptive algorithms for headway monitoring, illegal overtaking, over-speeding, and fatigue. The algorithms were then tested using a driving simulator. Results showed that the proposed adaptive headway warning algorithm was able to automatically update the headway warning thresholds and that, overall, the proposed algorithms provided the expected warnings. In particular, three or four different warning types were designed to distinguish different risk levels. The designed real-time intervention algorithms can be implemented in ADAS to provide warnings and interventions tailored to the driver status to further ensure driving safety.

1. Introduction

According to the Global status report on road safety 2018 [1], the number of road traffic deaths continues to rise steadily, reaching 1.35 million in 2016, and ranking as the eighth leading cause of death. The National Motor Vehicle Crash Causation Survey (NMVCCS) conducted from 2005 to 2007 indicated that the percentage of crashes involving driver error or impairment before the crash occurrence was as high as 94% [2]. Among them, recognition errors such as driver inattention, internal and external distractions, and inadequate surveillance, accounted for about 41% of crashes. Decision errors, including driving too fast in given situations (such as road design), false assumption of others’ actions including speed, and illegal maneuver and misjudgment of gaps, accounted for about 33% of the crashes [2]. The emergence of advanced driver assistance systems (ADASs) has therefore come to assist drivers in reducing or even eliminating driver errors, as they have been shown to overall improve driving safety [3]. The aim is to improve safety using automated technology, such as sensors and cameras, that can detect nearby obstacles or driver errors, and respond accordingly. Among many warning systems, popular ones include warnings and monitoring systems for headway, over-speeding, fatigue, and illegal overtaking.

The headway warning uses a forward-facing device such as a radar, Light Detection and Ranging (LiDAR), or a camera, to detect obstacles (e.g., car or pedestrian) in the path of the vehicle in which it is installed [4]. It monitors the distance (i.e., time headway) from the vehicle ahead, to provide alerts when the following time headway is below a pre-defined threshold, to help mitigate rear-end crashes [5]. This assists in significantly improving road traffic, as rear-end crashes are reported to account for more than 30% of crashes involving vehicles [5,6]. Similarly, over-speeding warning, fatigue warning, and illegal overtaking warning, monitor vehicle speeds, driver fatigue statuses, and overtaking maneuvers, respectively. Generally, as in any other ADAS warning, different types of warnings can be triggered once the predetermined thresholds have been exceeded. To the best of the authors’ knowledge, however, there is very limited literature that focuses on the implementation frameworks of these warnings, especially for over-speeding and illegal overtaking. Moreover, any related products in the market commonly use fixed and deterministic warning thresholds for all drivers (regardless of their status). For instance, [7] proposed to use 2.0 s as a cautionary threshold and set 1.5 s as a warning threshold for the visual representation of time headway. Generally, 5 km/h, 10 km/h, 15 km/h, 20 km/h, etc., are widely used as the thresholds for over-speeding warning in the real world or related products. These implementations or products therefore do not consider the impact of contributing factors, including weather, environment, types of vehicles around, risky hours, time of the day, fatigue, distraction, and drowsiness, on the driver operation and the traffic safety. This will occasionally lead to the issuance of annoying inadequate warnings and further reduce the acceptance and the effectiveness levels of ADAS [8].

Therefore, this paper aims to integrate the essential contributing factors into the warning algorithms, in an adaptive way that could automatically update them based on real-time traffic environments and driver status (such as distraction and fatigue), to capture driver diversity and changing parameters. This will be performed conceptually first, and then benefiting from an existing case study of a naturalistic driving experiment, the i-DREAMS case, in which the algorithms will be developed, tested, and validated.

The rest of this paper is organized as follows. The second section looks at related work in relation to ADAS and warning implementation. Afterwards, the case study is presented, including the ADAS system to be investigated, along with its sensory inputs. Thereafter, the detailed algorithms as well as their warning visualizations are proposed and detailed. After that, the validation of these algorithms is conducted by means of a driving simulation test, followed by a presentation of the results. Finally, the findings are discussed with the main insights extracted from the paper.

2. Related Work

Previous research has long used time headway as a parameter to develop forward collision warning frameworks; still, only a few studies focus on the implementation frameworks of headway warning and elaborate the thresholds that should be used in such systems. Among those are the studies [9,10], which use 0.9 s, 1.1 s, 1.6 s, 2.4 s as headway thresholds for different warning levels, in which 2.4 s could be replaced with the universally recommended headway of 2.0 s for dry roads. However, the thresholds of headway warning in these studies are deterministic and cannot satisfy driver variation and behavioral changes. Therefore, adaptive warning thresholds could come as a way to mitigate this limitation, as such warnings could be automatically updated based on the real-time time headway. For example, [11] proposes a forward collision warning (FCW) algorithm that can adjust its warning thresholds in a real-time manner according to driver behavior changes, including both behavioral fluctuation and individual differences. This adaptive FCW algorithm overcomes the limitations of traditional FCW with fixed risk evaluation models and fixed triggering thresholds by continuously monitoring driver braking behaviors in multiple lanes. Ref. [12] also proposed an adaptive FCW method that generates the warnings by continuously comparing time headway with a flexible threshold. The core of the proposed threshold updating mechanism is a real-time monitoring of the driver reactions against the previously generated warnings using the available indicators such as its braking history and driver distraction. Ref. [13] proposes the personalized threshold that is the mean of the minimal values of time headway, using at least 10 car following events, and ranging between 0.7 s and 2.0 s. However, some other important factors such as weather, environment, risky hours, time of the day, fatigue, etc., have not been considered in this updating mechanism.

As for the over-speeding warning, previous studies (e.g., [14]) have also not considered the warning thresholds. With regard to illegal overtaking, and to the best of the authors’ knowledge, limited to no literature exists; instead, research highlights how ADAS can provide guidance to the driver in making a safe overtaking maneuver based on the gap available between two successive opposing vehicles ([15,16,17]). Regarding fatigue warnings, previous studies focused on using advanced techniques such as Dynamic Bayesian Network [18], facial recognition technology [19], and speech-adapted pattern recognition approach [20], to detect driver fatigue. Ref. [21] proved that both truck and taxi drivers have a positive attitude towards fatigue warning systems. For instance, ref. [22] presented both qualitative and quantitative guidelines for designing drowsy-driver detection systems in a probabilistic framework based on the paradigm of Bayesian networks. Ref. [23] described a real-time online prototype driver-fatigue monitor framework that uses remotely located charge-coupled-device cameras equipped with active infrared illuminators to acquire video images of the driver. However, these studies have not discussed the thresholds of fatigue warnings.

Therefore, filling this gap becomes crucial; in particular, developing adaptive algorithms for headway warnings and over-speeding warnings and fine-tuning them based on real-time traffic environment and driver status, considering important contributing factors including weather, environment, risky hours, time of the day, fatigue, distraction, and drowsiness. When it comes to illegal overtaking warning framework, this needs to consider vehicle motion states. Finally, fatigue warnings need to consider the monitoring metrics of driver statuses.

3. Case Study: The i-DREAMS System

3.1. Context

The integrated advanced driver assistance system (ADAS) proposed in this paper is the one developed for the European naturalistic driving study (i-DREAMS), which included driving simulator and on-road trials for drivers in five countries (Germany, Belgium, Greece, Portugal, and the UK), in four modes (buses, trucks, cars, and rail), with the aim to define a safety-tolerance-zone to keep drivers in safe boundaries, based on real-time and post-trip interventions. For the real-time interventions, the aim was to propose a real-time algorithm methodology for the different warnings of interest. This paper discusses the design of those algorithms, as they were developed and implemented in the scope of this project. The data collection system described in the following sections relies on the technology developed in the scope of i-DREAMS and is the basis for the sensor data collected and used for the design of the algorithms. The four above-mentioned warning algorithms (for headway monitoring, over-speeding, illegal overtaking, and fatigue) run in parallel based on the sensor inputs including vehicle motion monitoring, vehicle speed monitoring, weather monitoring, distraction monitoring and fatigue monitoring. The output of these warnings is the visual signal warnings that are displayed in-vehicle for the drivers.

The technology is comprised of different components; the central one, however, is the gateway which gathers and centralizes information from the other components and handles data connectivity and transmission. The vehicle motion monitoring includes the Mobileye system, gateway accelerometer and gyroscope sensors. The Mobileye system can extract headway monitoring information, detect vehicles ahead, trigger urban forward collision warning, trigger involuntary lane departure warning, detect pedestrians ahead, and trigger pedestrian collision warning [24]. Moreover, it detects traffic signs in real-time, e.g., speed limit indication and forbidden overtaking signs. The system also reads information from the vehicle Controller Area Network (CAN) and produces a low visibility indicator. The vehicle speed monitoring is based on the Global Positioning System (GPS), and the Mobileye system. The GPS chip provides geostationary satellite localization services (GNSS), including speed, and vehicle heading in degrees. The weather monitoring is based on Mobileye system and web weather services (if possible). The distraction monitoring is based on the OSeven application (O7APP), which detects mobile phone use. During the driving, the O7APP records data via the O7SDK from the smartphone sensors, including the distraction information caused by mobile use (e.g., talking, texting, and internet navigation). The fatigue monitoring is based on the CardioWheel or a wristband, that collects the driver’s electrocardiogram (ECG), which enables the computation of heart rate variability parameters and provides an estimator for sleepiness. Figure 1 illustrates the architecture flow diagram including the sensor input, the four algorithms described in this paper, and the visual outputs displayed for the drivers.

3.2. Sensor Inputs

The input data for the above-described ADAS relies on sensory technology, such as Mobileye, Global Positioning System (GPS), CardioWheel and OSeven application (O7APP), to detect the traffic environments, driver behavior and driver status in real time. The data extracted from the different sensors are summarized and described in

Table 1. List of variables in proposed ADAS.

Source	Variable	Description
Mobileye (AWS)	THW	Time headway (float, second)
	Time_indicator	Time of day indicator (str): day, dusk, night
	Speed_limit	Speed limit sign recognition
Mobileye (Car)	Wiper_weather	Wipers indicator (bool): 1—on, 0—off
	Brake	Braking indicator (bool): 1—on, 0—off
	Speed	Vehicle speed (int): km/h
	Left_turn	Left turn signal indicator(bool): 1—on, 0—off
	Right_turn	Right turn signal indicator(bool): 1—on, 0—off
GPS	$\theta$	Vehicle heading in degrees (float)
O7APP	Distraction	Distraction (via hand-held mobile phone use): 1—use, 0—not use
CardioWheel	KSS	Karolinska Sleepiness Scale (int): −1 (invalid), 1 = extremely alert, 2 = very alert, 3 = alert, 4 = rather alert, 5 = neither alert nor sleepy, 6 = some signs of sleepiness, 7 = sleepy, but no effort to keep awake, 8 = sleepy, some effort to keep awake, 9 = very sleepy, great effort keeping awake, fighting sleep
Gateway	Driving_duration	Driving duration (hour)
Questionnaire	Age	Driver age (year)
	Gender	Driver gender(bool): 0—male, 1—Female
	Professional_driver	Professional driver (bool): 0—No, 1—Yes

.

It should be noted that Wiper_weather refers to the weather condition (i.e., rainy, not rainy) using wiper actions. When the wiper is non-transiently 1 (on), it indirectly shows the weather is rainy. Conversely, when the wiper is 0 (off), it indirectly shows the weather is not rainy.

4. Proposed Algorithms

4.1. Adaptive Headway Warning Algorithm

While previous studies already looked at innovative approaches combining well-known surrogate safety measures [25], these do not particularly focus on updating the algorithms in a more adaptive way. In this scope, this paper proposes an algorithm to update the headway warning thresholds. The proposed Algorithm 1 is given below:

Algorithm 1. Adaptive Headway Warning Framework

At any time instant t: Warning Generation Sub-System: $\mathrm{if}THW\left(t\right)\mathrm{i}\mathrm{s}\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{g}$ then warning_headway = −1 $\mathrm{else}\mathrm{if}THW\left(t\right)>2.5$ then warning_headway = 0 $\mathrm{else}\mathrm{if}{Th}_t<THW\left(t\right)\le 2.5$ then warning_headway = 1 else if 0.6 $<THW\left(t\right)\le {Th}_t$ then warning_headway = 2 $\mathrm{else}\mathrm{if}THW\left(t\right)\le 0.6$ then warning_headway = 3 end

Threshold Update Sub-System: $\mathrm{if}\mathrm{w}\mathrm{a}\mathrm{r}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\_\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\mathrm{w}\mathrm{a}\mathrm{y}\left(t\right)==1$ & $brake\left(t\right)==1\&speed\left(t\right)>10$ then                      ${Th}_t={Th}_{t-1}+a_1({THW}_{t_{bs}}-{Th}_{t-1})$……….…….…….…… (1) $\mathrm{else}\mathrm{if}\mathrm{w}\mathrm{a}\mathrm{r}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\_\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\mathrm{w}\mathrm{a}\mathrm{y}\left(t\right)==2$$\&speed\left(t\right)>10\&0>a_{long}\left(t\right)>-2$ then                      ${Th}_t={Th}_{t-1}-a_2({Th}_{t-1}-{THW}_{avg})$ …………….………… (2) End $\mathrm{If}fatigue\left(t\right)==1$ then    ${∆}_t={\theta }_1{e}^{KSS\left(t\right)}+{\theta }_{2}driving\_duration\left(t\right)+{\theta }_{3}time\_indicator\left(t\right)+{\theta }_{4}speed\left(t\right)+{\theta }_{5}web\_weather\left(t\right)$…….(3)            ${Th}_t={Th}_{t-1}+{∆}_t-{∆}_{t-1}$$,(\mathrm{Note}:{∆}_0=0$) end $\mathrm{temp}={Th}_{t-1}$ $\mathrm{if}distraction\left(t\right)==1$ then            ${Th}_t=Maximum$ else if $distraction\left(t\right)==0$ then            ${Th}_t=\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}$ end $\mathrm{If}{Th}_t>Maximum$ then            ${Th}_t=Maximum$ $\mathrm{If}{Th}_t<Minimum$ then            ${Th}_t=Minimum$

The headway warning has four phases:

• Normal Phase (warning_headway = 0): no warnings when the headway is greater than 2.5 s.
• Dangerous Phase (warning_headway = 1): when the headway is between the 2.5 s and the updated threshold, a visual warning indicating the dangerous phase is displayed.
• Avoidable Accident Phase (warning_headway = 2): when the headway is between the updated threshold and 0.6 s, a visual warning indicating the avoidable accident phase is displayed.
• Unavoidable Accident Phase (warning_headway = 3): It is a quite dangerous phase once the headway is less than 0.6 s. A frequent visual warning with alerts is displayed. The updated threshold ranges from the maximum and minimum values, which are 2.0 s and 1.0 s, respectively, to consider the reaction time of drivers which is not the same for every driver, and it varies from less than 1.0 s to about 2.0 s [26].

It should be noted that the initialized threshold of headway for cars is set at 1.5 s, while the initialized threshold of headway for buses/trucks is set at 2.0 s. The headway threshold (${Th}_t$) determines the ranges of different headway warning levels and can be updated based on the proposed algorithm. If the driver brakes and the warning is generated by the algorithm, the driver‘s normal risk tolerance is higher than the current time headway (THW) value which causes the driver to brake.

The first situation is when the driver’s brake is not accompanied by an application warning for the dangerous phase. In this situation, ${Th}_t$ needs to be updated increasingly according to Equation (1). In Equation (1), ${THW}_{t_{bs}}$ is the time headway at the starting moment of the current braking action $[t_{bs},t]$.$a_1$ is positive and plays the role of tuning coefficients for the ${Th}_t$ adjustment. The condition on $\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{e}\mathrm{d}>10\mathrm{k}\mathrm{m}/\mathrm{h}$ is set to exclude extreme and low speed situations ($\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{e}\mathrm{d}<10\mathrm{k}\mathrm{m}/\mathrm{h}$), such as the stop-and-go and going in and out of parking spaces, for which the adaptive warning algorithm is not useful. If the speed reduces due to braking or the effect of releasing the accelerator pedal while the system is generating the warnings for avoidable accident phase, which means that the current driver’s normal risk level is lower than the present value of ${Th}_t$, the threshold ${Th}_t$ should be updated according to Equation (2). In Equation (2), $a_2$ is positive while $a_{th}=-2 \mathrm{m}/{\mathrm{s}}^2$ is a negative constant ensuring the comfort braking deceleration [12]. The condition on the longitudinal acceleration $0>a_{long}>-2\mathrm{m}/{\mathrm{s}}^2$ is set to exclude the hard braking situations, since ${Th}_t$ should not be updated in abrupt braking as it is a temporary driver reaction in which the driver’s normal risk level is not affected. In addition, ${THW}_{avg}$ represents the average of $THW\left(t\right)$ in $[t_{bs},t]$.

If the warning is not generated, the threshold can be updated according to the distraction and fatigue levels. As distraction is quite dangerous, we can set the headway threshold as the maximum value. The headway threshold can go back to its previous value once distraction ends. The algorithm further checks the driver fatigue level (i.e., $Fatigue\left(t\right)=1$). Some important factors including KSS score, driving duration, speed, weather, etc., are used to fine tune the ${Th}_t$ adjustment. Finally, ${\theta }_1,{\theta }_2,{\theta }_3,{\theta }_4,{\theta }_5$, the impact coefficients for of KSS, driving duration, time indicator (time of day), speed, and weather, respectively can be identified based on data-driven analysis. For instance, ${\theta }_1$ should be quite small, like 0.00001.

4.2. Illegal Overtaking Warning Algorithm

Generally, a reasonable overtaking maneuver includes four phases, i.e., a preparation phase, a lane-changing phase, a passing phase, and a lane-returning phase, similar to the four phases of a car overtaking a cyclist [27]. While previous studies have looked into lane changing behavior, particularly extreme trait behaviors [28] and disordered heterogeneous traffic conditions [29], few if any looked at it as part of an adaptive overtaking warning algorithm. Our proposed real-time illegal overtaking warning algorithm aims to deter drivers from making an illegal overtaking move. The different stages are depicted in Figure 2. Drivers changing their lanes should not influence the vehicles around them; if the vehicles around the driver (i.e., driver A in Figure 2) take actions such as reducing speed, avoiding operation, etc., the overtaking move would not be considered reasonable and safe.

(1) In the preparation phase, the left-turn signal light should be kept on for at least 3.0 s before the start of lane changing. It is important to provide enough information and preparations for other vehicles around. Otherwise, the overtaking move would be considered dangerous. It should be noted that this example is for driving on the right. As for the driving on the left (such as in the UK and/or other countries), the right-turn signal light is the relevant one for the preparation phase.

(2) In the lane-changing phase, the reasonable overtaking move should avoid the risk of colliding with vehicle ahead. The speed $v$ and acceleration $a$ at any timestamp $t$ have longitudinal and lateral components, i.e., $v_{long}^t,v_{lat}^t,a_{long}^t,a_{lat}^t$. At the timestamp $t$, the instantaneous displacements in the longitudinal and latitude directions are

$$d_{long}^t=v_{long}^t∆t+\frac{1}{2}{a}_{long}^t{∆t}^2$$

(4)

$$d_{lat}^t=v_{lat}^t∆t+\frac{1}{2}{a}_{lat}^t{∆t}^2$$

(5)

where $v_{long}^t=v^t\cos \left({\theta }^t\right)$, $v_{lat}^t=v^t\sin \left({\theta }^t\right)$, $a_{long}^t=(v^t\cos ({\theta }^t)-v^{t-1}\mathrm{c}\mathrm{o}\mathrm{s}({\theta }^{t-1}))/∆T$, $a_{lat}^t=(v^t\sin ({\theta }^t)-v^{t-1}\sin ({\theta }^{t-1}))/∆T$. ${\theta }^t$, ${\theta }^{t-1}$ are the angles between the vehicle head and the lane marker stripe at timestamp $t$ and $t-1$, which can be calculated with the help of the Mobileye Advanced Warning System and the GPS. $v^t$ and $v^{t-1}$ are the speeds at timestamp $t$ and $t-1$. $∆T$ is the time interval between $t$ and $t-1$.

At any time, $d_{long}^t$ should be less than the lowest safety distance between the heading vehicles, and $d_{lat}^t$ should be less than the remaining width of the target lane, which is $d-{\int }_{t0}^t{v}_{lat}^{t}dt$, where $d$ is the lane width and $t0$ is the start time of the overtaking move. Here, the $∆t$ can be considered as the safety reaction time of the driver. We can set $∆t$ as 0.6 s, as a lower-bound reaction time [30].

(3) In the passing phase, the possible unsafe factor is acceleration since a too big acceleration will cause the vehicle to slip or lose controls. According to [31], the safe threshold curve of the acceleration is

$$a_{threshold}=g\left(0.569+0.198{\left(\frac{v^t}{100}\right)}^2-0.592\left(\frac{v^t}{100}\right)\right)$$

(6)

where $v^t$ is the vehicle speed in km/h at timestamp $t$ and g is 9.8 m/s². If the acceleration is higher than the threshold, the situation is considered dangerous. Additionally, this the safe threshold curve is also used to monitor the whole overtaking move.

(4) In the lane-returning phase, firstly, the right-turn signal light should be kept on for at least 3.0 s before the start of lane returning, and then the reasonable lane returning should also avoid the risk of colliding the heading vehicle and rushing out of the initial lane. For the situation of driving on the left, we consider the left-turn signal light in the lane-returning phase.

The algorithm of illegal overtaking warning is listed below. The illegal overtaking warning has four phases:

• Normal Phase (warning_overtaking = 0);
• Dangerous Phase (warning_overtaking = 1);
• Avoidable Accident Phase (warning_overtaking = 2);
• Unavoidable Accident Phase (warning_overtaking = 3).

If the time duration of keeping the turning light (i.e., left-turn signal light or right-turn signal light) on is less than 3.0 s and the absolute value of the heading degree changing is higher than $1.5°$, which is dangerous, the type 1 of warnings will be triggered. If the time duration of keeping the turning light on is less than 3.0 s and the vehicle touches the lane marker stripe, which is more dangerous, the type 2 of warnings will be triggered. If the acceleration is greater than the threshold, which is also dangerous, the type 2 warnings will be triggered. If the instantaneous displacements in the latitude directions $d_{lat}^t$ are greater than the rest width of the target lane, which is quite dangerous, type 3 of warnings will be triggered. Additionally, the longitudinal direction $d_{long}^t$ is detected and triggered according to the headway warning strategy. The proposed Algorithm 2 is presented below.

Algorithm 2. Illegal Overtaking Warning Framework

At any time instant t: Warning Generation Sub-System: warning_overtaking = 0 if $t_{light\_on}<3\&\mathrm{a}\mathrm{b}\mathrm{s}\left(\theta \right)>1.5°$ then warning_overtaking = 1 end if $t_{light\_on}<3\&\mathrm{d}==0$ then warning_overtaking = 2 else if $a^t>a_{threshold}$ then warning_overtaking = 2 end if $d_{lat}^t>d-{\int }_{t0}^t{v}_{lat}^{t}dt$ then warning_overtaking = 3 end

Figure 3 shows the change in max theta, latitudinal variables, and current speeds over time in the lane-changing phases for different initial speeds ($a$ = 0.5 m/s²) (see Equation (5)). It is noted that the color ranking of these curves in Figure 3 is the same with that in Figure 3a. The area between the curves and the x-axis in Figure 3a are the theta values for safe lane changings during the overtaking move. Regarding each lane change in the overtaking maneuver, the safe max theta has to reduce gradually over time or the latitude distance and speeds and can even be negative for a short time. After that, the theta is supposed to tend to zero. In addition, the higher the speed is, the lower the safe max theta is (see Figure 3a,f).

Figure 4 shows the relationship between safe acceleration and speed. The area under the curve is the acceleration values for a safe overtaking move. With the increase in the speed, the acceleration threshold reduces and tends to 1.2 m/s². If the acceleration is too high, the vehicle would easily skid and result in a road crash.

4.3. Over-Speeding Warning Algorithm

Over-speeding is the driving at a speed above the posted speed limit [32] and has bad impacts on traffic safety. Based on relevant literature (e.g., [33,34,35]) and traffic rules, this paper proposes threshold values for over-speeding under ideal conditions. In a good driving environment, we propose the different driving safety phases as follows:

• Normal Phase: driving speed < 0% above legal speed limit (SL);
• Dangerous Phase: driving speed = 0–5% over legal speed limit (SL);
• Avoidable Accident Phase: driving speed = 5–10% over legal speed limit (SL);
• Unavoidable Accident Phase: driving speed > 10% over legal speed limit (SL).

There are three thresholds, i.e., speed limit (SL), speed limit × (1 + 5%) = 1.05 SL, and speed limit × (1 + 10%) = 1.10 SL. The threshold is used as such for the unavoidable accident phase as in most countries it is not legal to drive at a speed 10% higher than the speed limit. It should be noted that these thresholds are not suitable in Germany since driving merely 3 km/h or faster above the posted or implied speed limit is considered a punishable infraction in Germany. Therefore, similarly to this kind of situation, we can use the speed limit × (1 − 10%) = 0.9 SL, speed limit × (1 − 5%) = 0.95 SL, and speed limit (SL) as the thresholds.

In the unideal conditions, the thresholds of the normal phase and dangerous phase (i.e., SL, 1.05 SL) need to be modified based on the impact factors:

The thresholds of normal phase and dangerous phase = Normal thresholds × Adjustment coefficient

(7)

Such unideal conditions, and the resulting adjustment coefficient, depend on factors related to environment (i.e., weather, risky hours, and time indicator) and to drivers themselves (i.e., fatigue, and distraction). Firstly, the adjustment value is the weighted average value of included factors for each of the environment and the driver, first considering equal weights. The distribution of the three factors (i.e., human, vehicle, and environment) has been shown to be 95.4%, 14.8%, 44.2% according to previous studies [36]. Accordingly, if we scale these values, we obtain the following percentages of 61.79%, 9.59%, 28.63%, for the driver, vehicle, and environment, respectively. Therefore, the final adjustment coefficient considers the weighted values of the environment and human drivers, whose weights are 28.63% and 61.79%. The proposed impact factors described above (and which would affect the adjustment coefficient) are listed in

Table 2. Impact factors for each one of risk factors under ideal conditions.

Items	Factors		Adjustment Coefficient
Environment factors	Web_weather	clear	0
		rain	−3.0%
		snow	−4.0%
		frost	−2.0%
	Wiper_weather	wiper_on	−3.0%
		wiper_off	0
	Risky hours	driving in risky hours 00:00 a.m.–05:00 a.m.	−3.0%
	Time_indicator	daytime	0
		dusk	−2.0%
		night time	−2.5%
Human factors	Fatigue	No tired (KSS $\le$ 5 or Driving_duration < 4.5 h)	0
		Tired (6 $\le$ KSS $\le$ 7 or (4.5 h $\le$ Driving_duration < 6 h))	−2.5%
		very tired (KSS $\ge$ 8 or Driving_duration $\ge$ 6 h)	−4.0%
	Distraction	Not distracted	0
		Distracted	−4.5%

. The adjustment coefficient of each impact factor ranges from −4.5% to 0 since the updated thresholds of the dangerous phase should not be less than the original speed limit value. Several important criteria, including the impact of factors on traffic safety and the ranking of factors contributing to crashes, are integrated synthetically to determine the adjustments coefficient of each factor.

4.4. Fatigue Warning Algorithm

The proposed real-time fatigue warning algorithm includes three warning levels based on the Karolinska Sleepiness Scale (KSS) score and the driving duration indicators. The three levels of warning are for the normal phase, the dangerous phase, and the avoidable accident phase. The KSS score uses three scores to determine the level of fatigue warning, while the driving duration indicator utilizes two thresholds (i.e., T1, T2) to divide the level of the fatigue warning. It is noted that, in this paper, fatigue is defined as the inability to continue with a task that has been continuing for too long [37] and can be influenced by monotony, workload, and task duration. The proposed fatigue warning strategy for drivers is developed as follows (Algorithm 3):

Algorithm 3. Fatigue Warning Framework

At any time instant t: Warning Generation Sub-System: $\mathrm{if}\mathrm{K}\mathrm{S}\mathrm{S}\left(t\right)\&\mathrm{D}\mathrm{r}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{n}\mathrm{g}\_\mathrm{d}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(t\right)\mathrm{i}\mathrm{s}\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{g}$ then warning_fatigue = −1 end $\mathrm{If}\mathrm{K}\mathrm{S}\mathrm{S}\left(t\right)\le 5\mathrm{o}\mathrm{r}\mathrm{D}\mathrm{r}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{n}\mathrm{g}\_\mathrm{d}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(t\right)<\mathrm{T}1$ then              warning_fatigue = 1 end $\mathrm{If}6\le \mathrm{K}\mathrm{S}\mathrm{S}\left(t\right)\le 7\mathrm{o}\mathrm{r}\mathrm{T}1\le \mathrm{D}\mathrm{r}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{n}\mathrm{g}\_\mathrm{d}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(t\right)<\mathrm{T}2$ then              warning_fatigue = 2 end $\mathrm{If}\mathrm{K}\mathrm{S}\mathrm{S}\left(t\right)\ge 8\mathrm{o}\mathrm{r}\mathrm{T}2\le \mathrm{D}\mathrm{r}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{n}\mathrm{g}\_\mathrm{d}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(t\right)$ then              warning_fatigue = 3 end

Threshold Update Sub-System: T1 = 3 h T2 = 4.5 h If driver is not a professional driver            T1 = T1 × 0.9            T2 = T2 × 0.9 end If $gender==female$ then            T1 = T1 × 0.95 end $\mathrm{If}Age\ge 60$ then            T1 = T1 × 0.9            T2 = T2 × 0.9 end

The initial thresholds (i.e., T1 and T2) are 3 h and 4.5 h, respectively, partially since professional drivers have to take an uninterrupted break of at least 45 min after a driving period of 4.5 h according to the European Union. These two thresholds can be further updated according to gender and age of drivers when taking into account that older drivers and female drivers are potentially less fit physically. Therefore, 0.95 is used to slightly update the thresholds for female drivers and 0.9 is applied to update slightly the thresholds for old drivers whose age is higher than 60 years.

4.5. Warning Visualizations

This paper also presents the warning visualizations for the different warning types of proposed warning algorithms.

Table 3. Warning visualizations for proposed headway warning, over-speeding warning and fatigue warning algorithms.

Warning Levels	Headway Warning	Over-Speeding Warning	Fatigue Warning
Normal phase	A green car	A speed limit sign with the current speed value in green.	No interventions
Dangerous phase	A yellow car with the time headway value in yellow.	A speed limit sign with the current speed value in yellow.	A yellow coffee symbol with the current driving duration value in red and auditory alarms.
Avoidable accident phase	A red car with the time headway value in red and auditory alarms.	A speed limit sign with the current speed value in red and auditory alarms.	A fatigue warning sign with increased pitch auditory alarms
Unavoidable accident phase	A red car with the time headway value in red and increased pitch auditory alarms.	A speed limit sign with the current speed value in red and increased pitch auditory alarms.

Note that the headway warning is triggered when the headway < 4.0 s, the over-speeding warning is triggered when the speed > speed limit values—20 km/h.

and

Table 4. Warning visualizations for the proposed illegal overtaking warning algorithm.

Warning Types	Description	Illustration Example
Normal phase	An overtaking warning sign.
Dangerous phase	An overtaking warning sign with a duration limit sign of the left-turn signal light on (i.e., 3 s), a flashing left turn sign and the current duration value of the left-turn signal light on in yellow.
	An overtaking warning sign with a duration limit sign of the right-turn signal light on (i.e., 3 s), a flashing right turn sign and the current duration value of the right-turn signal light on in yellow.
Avoidable accident phase	An overtaking warning sign with a duration limit sign of the left-turn signal light on (i.e., 3 s), a flashing left turn sign, the current duration value of the left-turn signal light on in red and auditory alarms.
	An overtaking warning sign with a duration limit sign of the right-turn signal light on (i.e., 3 s), a flashing right turn sign, the current duration value of the right-turn signal light on in red and auditory alarms.
	An overtaking warning sign with an acceleration limit sign, the current acceleration value in red and auditory alarms.
Unavoidable accident phase	An overtaking warning sign with a heading degree limit sign and the current heading degree value in red and increased pitch auditory alarms.

listed warning visualizations for the proposed warning algorithms. There are three levels for the fatigue warning strategy. Auditory alarms and increased pitch auditory alarms are applied for the dangerous phase and the avoidable accident phase to warn drivers. In addition, there are four levels for headway warning, over-speeding warning and illegal overtaking warning algorithms. Yellow (visual) and auditory alarms are applied for the avoidable accident phase to warn drivers. Red and increased pitch auditory alarms are applied for the unavoidable accident phase to warn drivers to take measures immediately, such as reducing speed, and reducing the heading degree.

5. Algorithm Validation

5.1. Driving Simulation Test

This study implements the warning algorithms in a driving simulator to further test whether they can output the expected warnings. The driving simulator is the Cockpit Sim that carefully recreates the feeling of driving a real vehicle by using authentic vehicle parts and equipment. The 3 × 50 inch, 130° FOV visual system provides a realistic, high-resolution driving view. Additionally, its size is not big and it can fit inside an office room; Figure 5 presents the driving simulator set-up used for this experiment. While theoretically different warnings could be activated at the same time, the set-up in place only displayed one warning at a time; if more than one were activated, one warning would override the other, in order to prevent driver disturbance.

The driving simulator uses the STISIM Drive 3 software, which features an open architecture and can be programmed according to specific requirements. STISIM Drive^® is the result of over 40 years of driving simulation research. It is used by over 500 universities, government agencies, medical facilities, training centers and corporations worldwide.

A highway without speed limits was created for testing the headway warning, illegal overtaking warning, and fatigue warning strategies. The highway has three lanes in each direction and lane width of 4.0 m. The total length of the highway section is 3.6 km. A rural road whose speed limits are 50 km/h and 70 km/h was also created to test the over-speeding warning strategy. It has two lanes in each direction and the lane width is also 4.0 m. The total length of the rural road is 4.4 km.

5.2. Results

Figure 6 shows the changing of time headway and headway thresholds according to the adaptive headway warning algorithm. It should be noted that the driver was asked to have more braking actions to have more situations to update the threshold and test the algorithm. In real-word driving, there would not be such highly frequent braking behavior. Additionally, the updating based on the fatigue and the distraction are not included since the parameter needs to be further identified by the data-driven analysis. In this test, $a_1$ was found to be 0.05 and $a_2$ 0.06.

The green, yellow, magenta and red are for the normal phase (headway > 2.5 s), dangerous phase (updated threshold for AA phase < headway ≤ 2.5 s), avoidable accident (AA) phase (0.6 s < headway ≤ updated threshold for AA phase) and unavoidable accident phase (headway ≤ 0.6 s), respectively. We can find that the updated threshold is changing over the occurrence of braking situations between 1.0 s and 2.0 s. The braking action in the dangerous phase will make the threshold lightly bigger in the dangerous phase based on Equation (1) and the bigger $a_1$ is, the bigger the improvement is. On the contrary, the low deceleration action in the avoidable accident phase will make the threshold lightly smaller based on Equation (2) and the bigger $a_2$ is, the bigger the decrease is.

Figure 7 illustrates a test example of the real-time illegal overtaking warning algorithm. The lateral position is the vertical distance to the center of dividing lines. The lateral velocity ($v_{lat}$) and lateral acceleration ($a_{lat}$) are the vertical components of the velocity and acceleration ($a$), respectively. The positive values are shown in the right direction while the negative values in the left direction. The heading degree ($\theta$) is the angle between the vehicle head and the lane marker stripe. The positive heading degree ($\theta$) presents that the vehicle heads to the right direction with respect to the direction of the lane marker stripe. The negative heading degree ($\theta$) presents that the vehicle heads to the left direction with respect to the direction of the lane marker stripe. The green, yellow, magenta and red are for the normal phase, dangerous phase, avoidable accident phase and unavoidable accident phase, respectively. The blue curve presents the acceleration threshold. It is labeled as the avoidable accident phase (i.e., magenta) when the acceleration is higher than the threshold.

By comparing the lateral velocity ($v_{lat}$) and the heading degree ($\theta$), we find that their curve tends are the same and that they have the same sign at the same location. The lateral velocity reduces or increases with the decrease or improvement of the heading degree. These two indicators achieve local extreme points (e.g., maximum, minimum, and zero) at the same location. On the contrary, the curve tends of the lateral velocity ($v_{lat}$) and the lateral acceleration ($a_{lat}$) are not the same. The lateral velocity achieves local maximum or minimum values when the lateral acceleration is zero temporarily. According to the lateral position, we can find that there are five overtaking moves in Figure 7. An overtaking move includes a preparation phase, a lane-changing phase, a passing phase, and a lane-returning phase. In the lane-changing phase, the heading degree ($\theta$) and the lateral velocity ($v_{lat}$) firstly starts to reduce until a negative local minimum value, and then increases until around zero. Meanwhile, the lateral acceleration ($a_{lat}$) firstly starts to reduce until a negative local minimum value, and then increases until a positive local maximum value, and then reduces to around zero. In the lane-returning phase, the heading degree ($\theta$) and the lateral velocity ($v_{lat}$) firstly starts to increase until a positive local maximum value, and then increases until around zero. Meanwhile, the lateral acceleration ($a_{lat}$) firstly starts to increase until a positive local maximum value, and then reduces until a negative local minimum value, and then improves to around zero. The start and end of their travelled distances of each overtaking move in Figure 7 are approximately (200 m, 645 m), (645 m, 930 m), (930 m, 1250 m), (1425 m, 1803 m) and (1803 m, 2384 m).

In Figure 7, some parts of the lane-changing phase and the lane-returning phase in the first overtaking move are labelled as the unavoidable accident phase (i.e., red) since the absolute value of their lateral velocities is too big. Similarly, some parts of the lane-changing phase in the second and fifth overtaking move are also labelled as the unavoidable accident phase (i.e., red). Since the lateral velocity mostly depends on the heading degree and velocity, it is therefore important to control the maximum value of the heading degree during the overtaking move.

Figure 8 illustrates a test example of the real-time over-speeding warning algorithm. The green, yellow, magenta, and red are for the normal phase (speed < threshold for dangerous phase), dangerous phase (threshold for dangerous phase ≤ speed < threshold for AA phase), avoidable accident (AA) phase (threshold for AA phase ≤ speed < threshold for UA phase) and unavoidable accident (UA) phase (speed ≥ threshold for UA phase), respectively. There are four kinds of driving conditions. They are, successively, a clear condition without the fatigue and distraction (time: 0–630.90 s), a night condition without the fatigue and distraction (time: 630.95–1230.05 s), a rainy night condition with a phone distraction (time: 1230.10–1840.75 s), and a rainy condition with a phone distraction (time: 1840.80–2500 s). According to Table 2, the adjustment coefficients is calculated (see Figure 8) and then the thresholds are updated with the help of the real-time over-speeding warning algorithm. Meanwhile, the speeding warning is provided in real time based on the current speed and thresholds.

Figure 9 illustrates a test example of the real-time fatigue warning algorithm. The green dashed line, yellow dotted line, and magenta solid line are for the normal phase, dangerous phase, and avoidable accident phase, respectively. The different levels of the fatigue warning are provided in real time with the change in the KSS and driving duration. It is noted that the first and second thresholds of the driving duration are 3 × 0.95 × 0.9 = 2.565 h, 4.5 × 0.9 = 4.05 h, respectively, since this is a non-professional young female driver. Therefore, it is identified as the dangerous phase (yellow) when the driving duration is greater than 2.565 h even though the KSS is still 5. Moreover, it is identified as the avoidable accident phase (magenta) during the 3.6 h and 4.05 h driving duration since the KSS is 8.

6. Conclusions

This paper proposed adaptive algorithms that could be automatically updated for each of headway monitoring, illegal overtaking, over-speeding, and fatigue based on real-time traffic environments and driver status, capturing thereby driver diversity and changing parameters, and filling the gap of existing deterministic and fixed-thresholds algorithms. Accordingly, this paper developed an integrated ADAS including the four above-mentioned warnings. These warning algorithms can fine-tune the thresholds based on real-time traffic environments and driver status, considering important contributing factors such as weather, environment, risky hours, time of the day, fatigue, distraction, and drowsiness, etc. Additionally, the proposed real-time illegal overtaking warning integrates the consideration of the lateral-orientation, safety, and instantaneous accelerations. In this work, we also visualized the change in max values of heading degree, latitude distance and current speeds over time in the lane-changing phases for different initial speeds. Furthermore, we implemented these warning algorithms in a driving simulator and further tested them. The results showed that this ADAS can provide the real-time warning of proposed warning algorithms. These algorithms are robust since both contextual and operator status variables are incorporated. The findings of this work are essential as they provide the exploratory simulation work needed to evaluate the behavior of different algorithmic possibilities and threshold values for the definition of a Safety Tolerance Zone [23] for different in-vehicle real-time warnings in preparation of final choices made in the real-world driving trips. Also, the results of this paper can be a potential framework of ADAS or in-vehicle real-time warning algorithms for industrial application.

Even though this paper promotes the improvement of ADAS largely, it does not come without limitations. While ADASs aim to assist and improve driving behavior, it is essential that future research considers the possible impact that such in-vehicle systems might have on driving behavior, as they might inflict themselves distraction on drivers, as previous research indicates [38]. With regard to the algorithm testing, due to cost and resource limitations and since the main paper objective was to develop the adaptive algorithms, these were only validated with tests driven by the authors themselves. Future research could look into extending the testing to a larger-scale experimental set-up, confirming the behavioral improvement resulting from the proposed algorithm (it is important to note that the algorithms proposed in this paper are different than the ones described in [39]). Finally, some important parameters also require to be trained and calibrated with the help of sensor data from different drivers in the real-world vehicle environment.