Driver Vision Based Perception-Response Time Prediction and Assistance Model on Mountain Highway Curve

To make driving assistance system more humanized, this study focused on the prediction and assistance of drivers’ perception-response time on mountain highway curves. Field tests were conducted to collect real-time driving data and driver vision information. A driver-vision lane model quantified curve elements in drivers’ vision. A multinomial log-linear model was established to predict perception-response time with traffic/road environment information, driver-vision lane model, and mechanical status (last second). A corresponding assistance model showed a positive impact on drivers’ perception-response times on mountain highway curves. Model results revealed that the driver-vision lane model and visual elements did have important influence on drivers’ perception-response time. Compared with roadside passive road safety infrastructure, proper visual geometry design, timely visual guidance, and visual information integrality of a curve are significant factors for drivers’ perception-response time.


Introduction
A variety of studies are under way today in developing driver assistance systems. A driver assistance system aims to help drivers to drive more safely under various traffic environments. Modern technologies, such as video detection, Light Detection and Ranging (LiDAR), and V2X communication, have been tested and used in some cases [1]. However, a new question arises with such rapid development of information-based systems: how do intelligent driving systems (including assistance driving systems) deal with these forms of information? Will they respond like a machine or a human being? Responding like a machine means using a pre-set unchangeable process mode as the core of the system without consideration of drivers' personal needs. A humanized system learns to respond in human recognition mode that takes both a driver's perspective and real-time mechanical data into consideration, which would thereby provide suitable assistance according to different driving conditions.
In this research study, an intelligent driving system is regarded as the interaction between a driver and a vehicle. The advanced driver assistance system (ADAS) has been improved a lot in recent years, with many features such as obstacle warning, collision warning, lane control and traffic light assistance strategy, to keep drivers informed of the latest information or warnings during the driving process [2][3][4][5]. In such systems, the so-called "Perception-response time" (PR time) or "Perception-response distance" is usually defined as an empirical value (e.g., 3 s) according to the Chinese Highway Standard [6] or a fixed range (e.g., 1.5 to 3.5 km) [7]. However, perception-response (PR) time as defined by Olson and Sivak is "the time from the first sighting of an obstacle until the driver applies the brakes" [8]. It was obtained by a driver's vision and the perception-response time on a mountain highway curve is discussed. Based on on-board video, Catmull-Rom spline describes the driver-vision lane and corresponding elements are extracted. These visual factors and mechanical factors are then used in a multinomial log-linear model via neural network, which has also been verified with a series of field tests. This work serves as the basis of human-like ADAS that takes actual perception-response time into consideration. This paper consists of five sections as shown in Figure 1.

Definition of Perception-Response Time on Mountain Highway Curves
Traditional perception-response time on a highway according to the Chinese Highway Standard is divided into three parts: driver's sensing time, reaction time, and braking time. The first two parts show the human factors in perception-response time. A mechanical process is considered in the third part. It is always considered as a value (3 s) in the design and evaluation of a highway. However, highways with various combinations of characteristics/or different environmental conditions can result in different driving behaviors. For instance, drivers may take a variety of strategies (e.g., steering, lifting the foot from accelerator, or braking) when a highway curve appears, which will lead to different perception-response times.
In this study, the perception-response time on a mountain highway curve is defined as the duration from the first sight of a curve's appearance in a driver's sight to the moment of the driver's reaction (braking or turning). Figure 2 shows the meaning of this term. This identification is different from the perception concept in Embodied Cognition Theory [23], which reflects the overall response period from the curve appearance to the first detection moment of driver reaction (either steering or braking). The first section introduces the background and purpose of this study. The second section illustrates the definition of PR time in this research. The experiments data and visual information are also explained. The third section builds a multinomial log-linear prediction model for PR time. A corresponding assistance model is also established and tested in the fourth part. The last section concludes the key points of this study.

Definition of Perception-Response Time on Mountain Highway Curves
Traditional perception-response time on a highway according to the Chinese Highway Standard is divided into three parts: driver's sensing time, reaction time, and braking time. The first two parts show the human factors in perception-response time. A mechanical process is considered in the third part. It is always considered as a value (3 s) in the design and evaluation of a highway. However, highways with various combinations of characteristics/or different environmental conditions can result in different driving behaviors. For instance, drivers may take a variety of strategies (e.g., steering, lifting the foot from accelerator, or braking) when a highway curve appears, which will lead to different perception-response times.
In this study, the perception-response time on a mountain highway curve is defined as the duration from the first sight of a curve's appearance in a driver's sight to the moment of the driver's reaction (braking or turning). Figure 2 shows the meaning of this term. This identification is different from the perception concept in Embodied Cognition Theory [23], which reflects the overall response period from the curve appearance to the first detection moment of driver reaction (either steering or braking).

Curve Appearance React
Perception-response Time Time Straight Figure 2. Illustration of perception-response time on a mountain highway curve.

Experiments
To understand the impact of different perception-response (PR) times on driving behaviors on mountain highway curves, 220 field experiments were conducted during the daytime on mountain highway curves (design speed: 40 km/h) in Lishui and Suichang city, Zhejiang Province, China. Based on field conditions, tested curves were divided into six types. Detailed parameters are listed in Table 1. A total of 32 drivers who had over three years of driving experience (Mean: 11.4, S.D.: 2.4) took part in the experiments. They were between the age of 28 and 40 (Mean: 38.8, S.D.: 3.5). During the tests, they were not assigned with any task. Each driver faced the same route and were asked to drive

Experiments
To understand the impact of different perception-response (PR) times on driving behaviors on mountain highway curves, 220 field experiments were conducted during the daytime on mountain highway curves (design speed: 40 km/h) in Lishui and Suichang city, Zhejiang Province, China. Based on field conditions, tested curves were divided into six types. Detailed parameters are listed in Table 1.
A total of 32 drivers who had over three years of driving experience (Mean: 11.4, S.D.: 2.4) took part in the experiments. They were between the age of 28 and 40 (Mean: 38.8, S.D.: 3.5). During the tests, they were not assigned with any task. Each driver faced the same route and were asked to drive on the six types of curves respectively. A driving recorder (GARMIN GDR35) was fixed on windshield to record the front view. During the tests, a built-in sensor that was synchronized with the camera and fixed on the steering wheel recorded the vehicle's mechanical changes. Table 2 shows the detailed data collected by sensors.  According to the video and vehicle data, the perception-response time of each test can be obtained. After eliminating incomplete and wrong records, 129 tests were applied in the following study. The population distribution of perception-response time shown in Figure 3 indicates that most tested drivers were inclined to respond between 1 and 2 s. on the six types of curves respectively. A driving recorder (GARMIN GDR35) was fixed on windshield to record the front view. During the tests, a built-in sensor that was synchronized with the camera and fixed on the steering wheel recorded the vehicle's mechanical changes. Table 2 shows the detailed data collected by sensors. According to the video and vehicle data, the perception-response time of each test can be obtained. After eliminating incomplete and wrong records, 129 tests were applied in the following study. The population distribution of perception-response time shown in Figure 3 indicates that most tested drivers were inclined to respond between 1 and 2 s. However, different scenarios would have diverse impacts on drivers' perception-response times. A comparative analysis was done to reveal drivers' perception-response times on the six kinds of curves. Figure 4 shows the histogram and cumulative frequency curve [24][25][26] of the tested curves.
The average and standard deviation of the PR times show that drivers reacted quickly on near curves, especially on near right curves. The variation of PR times on near right curves is also quite low. This means most drivers' perception-response times on such curves are relatively consistent. However, the middle curves, including right and left curves, tended to result in longer PR times than other types of curves, which was not expected. It can also be seen from Figure 4 that 85% of drivers reacted within two seconds when faced with a near curve on a mountain highway. On middle-right curves, over 30% of participants' PR times were among 1~2 s and the 85th quantile of PR times was over 3 s. The test results also show that drivers' PR times were usually under 3 s (0~3 s) on middle-left curves and far-right curves. On far-left curves, they tended to react within 1~3 s. Three mechanical coefficients of variation on six types of curves during the turning process are listed in Table 3.
where σ is standard deviation; μ is average value. However, different scenarios would have diverse impacts on drivers' perception-response times. A comparative analysis was done to reveal drivers' perception-response times on the six kinds of curves. Figure 4 shows the histogram and cumulative frequency curve [24][25][26] of the tested curves.
The average and standard deviation of the PR times show that drivers reacted quickly on near curves, especially on near right curves. The variation of PR times on near right curves is also quite low. This means most drivers' perception-response times on such curves are relatively consistent. However, the middle curves, including right and left curves, tended to result in longer PR times than other types of curves, which was not expected. It can also be seen from Figure 4 that 85% of drivers reacted within two seconds when faced with a near curve on a mountain highway. On middle-right curves, over 30% of participants' PR times were among 1~2 s and the 85th quantile of PR times was over 3 s. The test results also show that drivers' PR times were usually under 3 s (0~3 s) on middle-left curves and far-right curves. On far-left curves, they tended to react within 1~3 s. Three mechanical coefficients of variation on six types of curves during the turning process are listed in Table 3.
Three kinds of coefficients were relatively low among 85% tested drivers, which is in accordance with the distribution of PR time. This means that a suitable PR time would result in good turning process (low CV). In the following assistance model, the 85th quantile of PR time will be regarded as the boundary value for the recommended range.
Three kinds of coefficients were relatively low among 85% tested drivers, which is in accordance with the distribution of PR time. This means that a suitable PR time would result in good turning process (low CV). In the following assistance model, the 85th quantile of PR time will be regarded as the boundary value for the recommended range.

Visual Information Extraction
According to our previous study (see Yu et al. [27]), the road alignment in a driver's vision can be described as a driver-vision lane (the blue line in Figure 5) during the process of driving. The driver-vision lane model is based on Catmull-Rom spline to fit with a driver's visual perception (a detailed explanation of this model is available in Yu et al. [27]). Perception characteristics of road alignment in this research include the following elements obtained by the driver-vision lane model.
where V is the visual curve length between control point Pi and Pi+1 (pixels); V is the visual curve curvature between control point Pi and Pi+1. There are two rules to judge a curve based on an image with the elements above: (1) Curve appearance: On a straight highway, the tangent direction angles are normally equal to each other. When there is a curve ahead, there will be a change in f4. (2) Right/left curve: f4 > 0 and V < 0 means right curve; f4 < 0 and V > 0 means left curve.
Apart from detecting the elements of the driver-vision lane model, we also classified the visual information of traffic/road environment into six major parts. Each part consists of four sub-classifications (see in Table 4).

Perception-Response Time Prediction Model
In this study, a driver's perception-respond (PR) time is predicted by three parts: traffic/road environment information, driver-vision lane model, and mechanical status (last second). Traffic/road There are four control points (P i , i = 1, 2, 3, 4) on Catmull-Rom spline. The value of 1 accumulated spline length (S i , i = 1, 2, 3); 2 spline length between P i and P i−1 (V S i , i = 1, 2, 3); 3 tangent direction angle of P i (f i , i = 1, 2, 3, 4); 4 curvature rate between P i and P i−1 (V K i , i = 1, 2, 3) are decided by P i . The relationship between these elements is as follows.
where V S i is the visual curve length between control point P i and P i+1 (pixels); V K i is the visual curve curvature between control point P i and P i+1 . There are two rules to judge a curve based on an image with the elements above: (1) Curve appearance: On a straight highway, the tangent direction angles are normally equal to each other. When there is a curve ahead, there will be a change in f 4 . (2) Right/left curve: f 4 > 0 and V K 3 < 0 means right curve; f 4 < 0 and V K 3 > 0 means left curve.
Apart from detecting the elements of the driver-vision lane model, we also classified the visual information of traffic/road environment into six major parts. Each part consists of four sub-classifications (see in Table 4).

Perception-Response Time Prediction Model
In this study, a driver's perception-respond (PR) time is predicted by three parts: traffic/road environment information, driver-vision lane model, and mechanical status (last second). Traffic/road environment is a uniform factor for each driver. It provides drivers with similar driving tasks. However, similar or even the same curve conditions did not result in the same PR times according to the tests. This means there are some other factors effecting drivers' PR processes, such as the driver-vision lane model. It is the impact of lane alignment on drivers, and last second speed and acceleration adjustment is another facet considered in the prediction model.
In Figure 6, the three parts all have a certain impact on drivers' PR times, for drivers' responses depend on some knowledge: what are the environmental conditions (traffic/road environment), what is the shape of the next curve (driver-vision lane model), and what is the status of the vehicle (mechanical status). Detailed parameters of these three parts are listed in Table 5.     Table 6.
The model provided here is a multinomial log-linear regression model based on one-hidden-layer neural network. The p-value is an index of the multinomial log-linear model, which is a post-calculated value. AIC is Akaike Information Criterion, which is used as a measure of the relative quality of statistical models for a given set of data. In this model, V K 2 and V K 3 of driver-vision lane have the most significant impact on drivers' PR times, especially on the PR times during 2~3 s and >3 s. Compared with V K 2 , V K 3 shows a greater determining effect on PR time. Although it shows a negative impact on the third category, one unit change (Odds Ratio [28]) of it would result in the most significant change on PR time compared with other factors. Hence V K 3 is a determining factor. A large V K 3 would lead to a sharp drop in drivers' PR times. Highway access is the third important factor which would result in lower PR times (<3 s). Among the protective roadside marking and infrastructure, warning pier is the most effective way to guide drivers. The existence of corrugated-steel guardrails has a negative impact on PR times over 1 s, so that they would accelerate the PR process. Concrete guardrails and traffic markings/signs have similar low influence on PR times. Each additional opposite vehicle results in additive negative effects. This means that a high volume of opposite traffic increases driving task difficulty and decreases PR times. All the distance of curve appearance parameters are positive, and they have larger impacts on longer PR time. This conforms to common knowledge. However, curve distance between 30 m-50 m has a greater positive influence on both short and long PR time compared with curve distance over 50 m. This reveals that drivers responded dispersedly in such circumstance. PR times increase with large longitudinal impact force (steep slope) and decrease with vertical impact force (rough road surface). This shows the impact of road geometry on drivers' PR times. Drivers' real-time operation consists of speed and acceleration changing rate, but they are not significant in the prediction model.  Note: a Perception-response time categories: 1 (1~2 s), 2 (2~3 s), 3 (>3 s); 0 (0~1 s) is the control group; b p-Value significance: "***": 0 ≤ p ≤ 0.001, "**": 0.001 < p ≤ 0.01, "*": 0.01 < p ≤ 0.05, " # ": 0.05 < p ≤ 0.1;; c "Distance" in the table means how far the curve is when it comes into view. It was classified into: >50 m, 30-50 m, <30 m, considered as dummy variables.

Model Structure
According to the PR time prediction model and the recommended range of PR times introduced in Section 2.2, this section will build a PR time assistance model to help drivers to respond timely and properly on a mountain highway curve. Figure 7 describes the flowchart of the assistance model.

Model Calibration
There were six tests conducted on each type of curve. Six drivers did one test on six types of curves respectively. The PR time prediction results are shown in Figure 8.
Curve Assistance Warning: Please keep the status until you get the next message.
Curve Assistance Warning: Please take turning operation now.   In Figure 8, the black bar stands for the tests of which the predicted PR time is inappropriate (beyond or below the recommended range). The dash line is the recommended range of PR time according to the 85th quantile in Figure 4. The PR times of most drivers were within this range. On near-left curves, no sample was beyond the recommended range. All cases (black bars) with unsuitable PR time were detected by the assistance model (see Table 7). Among the five curve types, the far-left curves tended to result in a large deviation on PR times, followed by the middle-right curve. This means that when a far-left curve appeared, some drivers would take reaction within 1 s and other drivers tended to respond far more slowly. Average dispersion change of CV had more improvement on right-turning curves than left-turning curves after assistance, especially on near-right curves. This would result from the fact that the right-turning process has more flexibility than left-turning process according to the traffic rule in China. The possibility of an unsafe turning process (large value of three CVs) would increase with different PR times. Hence proper assistance strategy can help drivers to behave well in such circumstances.

Conclusions
This study defined the concept of perception-response time (PR time) on mountain highway curves. Field tests were conducted with a driving recorder to record drivers' real-time driving data both before and during the turning process. As drivers' visual information is an important factor in perception-response time prediction in this study, driver-vision lane model was used to extract the elements of visual information from the sight of the drivers. To predict the perception-response times, a multinomial log-linear prediction model with the elements of traffic/road environment, driver-vision lane model, and mechanical status was presented. A corresponding assistance model was also illustrated to help drivers to have more proper perception-response times on mountain highway curves.
The results showed that the same road situation could result in various understandings regarding the interpretation of perception-response times in this work. Compared with the length of driver-vision lane, curvature rate (especially V K 3 ) had more impact on PR times. This indicates that drivers' PR process is mostly based on the visual information integrality of curves. Uncertainty of the curve shape and curvature rate will lead to uncertain PR times, such as observed with the middle curve (middle-VAR: 1.162, near-VAR: 0.609, far-VAR: 0.826). Meanwhile, the effect of road safety infrastructure are less significant than that of driver-vision lane elements. Therefore, good road visual geometry is more effective than passive road safety infrastructure in driving behavior guidance.
On the other hand, this study verified the need for establishing a personal-targeted assistance model, which took drivers' personal behavior and visual information into consideration. This is more humanized than a traffic-rule-based system. Although the model in this study has been proven valid, the absolute data and the meaning obtained should be considered only as relative result [29] due to the limitation of low-frequency data. More high-frequency and accurate biological information detectors would contribute to future studies. The change of eye gaze position/area would also help to illustrate drivers' different perception processes to establish a more personalized assistance system. In the future, complex driving behavior and driver-based decision processes will be analyzed and quantified.