Abstract
To make driving assistance system more humanized, this study focused on the prediction and assistance of drivers’ perception-response time on mountain highway curves. Field tests were conducted to collect real-time driving data and driver vision information. A driver-vision lane model quantified curve elements in drivers’ vision. A multinomial log-linear model was established to predict perception-response time with traffic/road environment information, driver-vision lane model, and mechanical status (last second). A corresponding assistance model showed a positive impact on drivers’ perception-response times on mountain highway curves. Model results revealed that the driver-vision lane model and visual elements did have important influence on drivers’ perception-response time. Compared with roadside passive road safety infrastructure, proper visual geometry design, timely visual guidance, and visual information integrality of a curve are significant factors for drivers’ perception-response time.
1. Introduction
A variety of studies are under way today in developing driver assistance systems. A driver assistance system aims to help drivers to drive more safely under various traffic environments. Modern technologies, such as video detection, Light Detection and Ranging (LiDAR), and V2X communication, have been tested and used in some cases [1]. However, a new question arises with such rapid development of information-based systems: how do intelligent driving systems (including assistance driving systems) deal with these forms of information? Will they respond like a machine or a human being? Responding like a machine means using a pre-set unchangeable process mode as the core of the system without consideration of drivers’ personal needs. A humanized system learns to respond in human recognition mode that takes both a driver’s perspective and real-time mechanical data into consideration, which would thereby provide suitable assistance according to different driving conditions.
In this research study, an intelligent driving system is regarded as the interaction between a driver and a vehicle. The advanced driver assistance system (ADAS) has been improved a lot in recent years, with many features such as obstacle warning, collision warning, lane control and traffic light assistance strategy, to keep drivers informed of the latest information or warnings during the driving process [2,3,4,5]. In such systems, the so-called “Perception-response time” (PR time) or “Perception-response distance” is usually defined as an empirical value (e.g., 3 s) according to the Chinese Highway Standard [6] or a fixed range (e.g., 1.5 to 3.5 km) [7]. However, perception-response (PR) time as defined by Olson and Sivak is “the time from the first sighting of an obstacle until the driver applies the brakes” [8]. It was tested that the PR time of over 95% of tested drivers, including young and old, was 1.6 s. This indicates that the value suggested by the current national standard is improper, due to the loss of real individual data. However, timely, relevant, and comprehensive information is quite vital when providing drivers with more choices and perception-response time to an emergency situation [9].
As PR time varies among individuals, personal driving behavior should to be included in order to meet drivers’ humanized demands. Past studies mainly treat this factor as driving experience, which appeared to be a safe-relevant element in comparative study. It showed a positive impact in drivers’ response time to static roadway hazards [10]. By estimating the performance of novice, experienced, highly-experienced, and taxi drivers through video and eye movement data, Borowsky and Oron-Gilad revealed that drivers who had more driving experience were more skilled in the awareness of materialized hidden hazards [11]. Moreover, skilled drivers alter their driving behavior more frequently than the unexperienced ones to adapt accordingly to changing driving situations [12]. Although more driving experience would help drivers to respond more properly than new drivers [13], its impact differs among drivers. It is not always important in all situations. According to Sagberg and Bjørnskau [14], driving experience did not show a strong relationship with hazard perception time. It also did not show a significant influence on obvious risk perception. Some detailed methods have been introduced to describe the effects of different driving behaviors, such as driving habit graph (DHG) which can be utilized to explain different driving performances [15]. Naturalistic driving factors are also considered in individual-targeted driving response time studies, including safety margin adjusting [16], braking control [17], and eye-off-road ratio [18]. Drivers’ gaze data were also used in ADAS to identify simulation information for drivers and to make predictions of behaviors [19,20].
The factors above reflect drivers’ performance and differentiation, but ultimately the performance of drivers is still largely based upon the perception and decisions of drivers. A driving assistance system based on these elements loses the “Perception-Response” process. It cannot really “think” as a real driver from the first identification of a changing driving circumstance. Human-simulated ADAS needs to respond like a human in advance, such as the graphical model proposed by Oliver and Pentland who established the relationship between driver and vehicle with vehicle data, road geometric elements, and drivers’ vision data [21].
This research, as a further study, focuses on mountain highway curves in China where most accidents happened in 2014 among three kinds of highways curves: plain (3875 accidents), hill (3981 accidents), and mountain (6038 accidents) [22]. Different from the definition of perception-response time (PRT) in Olson and Sivack’ theory, the PRT in this study focused on “left/right curve” rather than “obstacle”. The PRT in their theory is an important component of stopping sight distance on a vertical curve (e.g., cresting a hill) to avoid collision. However, in this research we predict drivers’ response time from the curve appearance until the first reaction, during which they do not need to stop, but to drive safely. For the different driving purpose, we do not apply their theory in our PRT definition or prediction model which is only serves as a hint or piece of evidence for us to improve Chinese current national standard. The inner relationship between road information obtained by a driver’s vision and the perception-response time on a mountain highway curve is discussed. Based on on-board video, Catmull-Rom spline describes the driver-vision lane and corresponding elements are extracted. These visual factors and mechanical factors are then used in a multinomial log-linear model via neural network, which has also been verified with a series of field tests. This work serves as the basis of human-like ADAS that takes actual perception-response time into consideration.
This paper consists of five sections as shown in Figure 1.
Figure 1.
Workflow of the research.
The first section introduces the background and purpose of this study. The second section illustrates the definition of PR time in this research. The experiments data and visual information are also explained. The third section builds a multinomial log-linear prediction model for PR time. A corresponding assistance model is also established and tested in the fourth part. The last section concludes the key points of this study.
2. Experiments and Data
2.1. Definition of Perception-Response Time on Mountain Highway Curves
Traditional perception-response time on a highway according to the Chinese Highway Standard is divided into three parts: driver’s sensing time, reaction time, and braking time. The first two parts show the human factors in perception-response time. A mechanical process is considered in the third part. It is always considered as a value (3 s) in the design and evaluation of a highway. However, highways with various combinations of characteristics/or different environmental conditions can result in different driving behaviors. For instance, drivers may take a variety of strategies (e.g., steering, lifting the foot from accelerator, or braking) when a highway curve appears, which will lead to different perception-response times.
In this study, the perception-response time on a mountain highway curve is defined as the duration from the first sight of a curve’s appearance in a driver’s sight to the moment of the driver’s reaction (braking or turning). Figure 2 shows the meaning of this term. This identification is different from the perception concept in Embodied Cognition Theory [23], which reflects the overall response period from the curve appearance to the first detection moment of driver reaction (either steering or braking).
Figure 2.
Illustration of perception-response time on a mountain highway curve.
2.2. Experiments
To understand the impact of different perception-response (PR) times on driving behaviors on mountain highway curves, 220 field experiments were conducted during the daytime on mountain highway curves (design speed: 40 km/h) in Lishui and Suichang city, Zhejiang Province, China. Based on field conditions, tested curves were divided into six types. Detailed parameters are listed in Table 1.
Table 1.
Categories of Tested Curves.
A total of 32 drivers who had over three years of driving experience (Mean: 11.4, S.D.: 2.4) took part in the experiments. They were between the age of 28 and 40 (Mean: 38.8, S.D.: 3.5). During the tests, they were not assigned with any task. Each driver faced the same route and were asked to drive on the six types of curves respectively. A driving recorder (GARMIN GDR35) was fixed on windshield to record the front view. During the tests, a built-in sensor that was synchronized with the camera and fixed on the steering wheel recorded the vehicle’s mechanical changes. Table 2 shows the detailed data collected by sensors.
Table 2.
Data Collected by Inner Sensor of Driving Recorder.
According to the video and vehicle data, the perception-response time of each test can be obtained. After eliminating incomplete and wrong records, 129 tests were applied in the following study. The population distribution of perception-response time shown in Figure 3 indicates that most tested drivers were inclined to respond between 1 and 2 s.
Figure 3.
Distribution of perception-response time (all tests).
However, different scenarios would have diverse impacts on drivers’ perception-response times. A comparative analysis was done to reveal drivers’ perception-response times on the six kinds of curves. Figure 4 shows the histogram and cumulative frequency curve [24,25,26] of the tested curves.
Figure 4.
Histogram and cumulative frequency curve of perception-response time on six types of curves.
The average and standard deviation of the PR times show that drivers reacted quickly on near curves, especially on near right curves. The variation of PR times on near right curves is also quite low. This means most drivers’ perception-response times on such curves are relatively consistent. However, the middle curves, including right and left curves, tended to result in longer PR times than other types of curves, which was not expected. It can also be seen from Figure 4 that 85% of drivers reacted within two seconds when faced with a near curve on a mountain highway. On middle-right curves, over 30% of participants’ PR times were among 1~2 s and the 85th quantile of PR times was over 3 s. The test results also show that drivers’ PR times were usually under 3 s (0~3 s) on middle-left curves and far-right curves. On far-left curves, they tended to react within 1~3 s. Three mechanical coefficients of variation on six types of curves during the turning process are listed in Table 3.
where σ is standard deviation; μ is average value.
CV = σ/μ
Table 3.
Average Value of Three Mechanical Coefficients of Variation (CV) on Six Types of Curves.
Three kinds of coefficients were relatively low among 85% tested drivers, which is in accordance with the distribution of PR time. This means that a suitable PR time would result in good turning process (low CV). In the following assistance model, the 85th quantile of PR time will be regarded as the boundary value for the recommended range.
2.3. Visual Information Extraction
According to our previous study (see Yu et al. [27]), the road alignment in a driver’s vision can be described as a driver-vision lane (the blue line in Figure 5) during the process of driving. The driver-vision lane model is based on Catmull-Rom spline to fit with a driver’s visual perception (a detailed explanation of this model is available in Yu et al. [27]). Perception characteristics of road alignment in this research include the following elements obtained by the driver-vision lane model. The bottom-left corner of the on-board driving recorder video is set as the origin of coordinate.
Figure 5.
Driver-vision lane model based on driver’s visual perception (unit: pixel).
There are four control points (Pi, i = 1, 2, 3, 4) on Catmull-Rom spline. The value of ① accumulated spline length (Si, i = 1, 2, 3); ② spline length between Pi and Pi−1 (, i = 1, 2, 3); ③ tangent direction angle of Pi (fi, i = 1, 2, 3, 4); ④ curvature rate between Pi and Pi−1 (, i = 1, 2, 3) are decided by Pi. The relationship between these elements is as follows.
where is the visual curve length between control point Pi and Pi+1 (pixels); is the visual curve curvature between control point Pi and Pi+1.
There are two rules to judge a curve based on an image with the elements above:
- (1)
- Curve appearance: On a straight highway, the tangent direction angles are normally equal to each other. When there is a curve ahead, there will be a change in f4.
- (2)
- Right/left curve: f4 > 0 and < 0 means right curve; f4 < 0 and > 0 means left curve.
Apart from detecting the elements of the driver-vision lane model, we also classified the visual information of traffic/road environment into six major parts. Each part consists of four sub-classifications (see in Table 4).
Table 4.
Visual Information Classification of Traffic/Road Environment.
3. Perception-Response Time Prediction Model
In this study, a driver’s perception-respond (PR) time is predicted by three parts: traffic/road environment information, driver-vision lane model, and mechanical status (last second). Traffic/road environment is a uniform factor for each driver. It provides drivers with similar driving tasks. However, similar or even the same curve conditions did not result in the same PR times according to the tests. This means there are some other factors effecting drivers’ PR processes, such as the driver-vision lane model. It is the impact of lane alignment on drivers, and last second speed and acceleration adjustment is another facet considered in the prediction model.
In Figure 6, the three parts all have a certain impact on drivers’ PR times, for drivers’ responses depend on some knowledge: what are the environmental conditions (traffic/road environment), what is the shape of the next curve (driver-vision lane model), and what is the status of the vehicle (mechanical status). Detailed parameters of these three parts are listed in Table 5.
Figure 6.
Three parts in perception-response time prediction model.
Table 5.
Explanation of Parameters Considered in the Prediction Model.
A multinomial log-linear model via a neural network can be established to quantify the weights of the parameters above. After 790 iterations, the converged model results are shown in Table 6.
Table 6.
Perception-Response Time Prediction Model on Mountain Highway Curves.
The model provided here is a multinomial log-linear regression model based on one-hidden-layer neural network. The p-value is an index of the multinomial log-linear model, which is a post-calculated value. AIC is Akaike Information Criterion, which is used as a measure of the relative quality of statistical models for a given set of data. In this model, and of driver-vision lane have the most significant impact on drivers’ PR times, especially on the PR times during 2~3 s and >3 s. Compared with , shows a greater determining effect on PR time. Although it shows a negative impact on the third category, one unit change (Odds Ratio [28]) of it would result in the most significant change on PR time compared with other factors. Hence is a determining factor. A large would lead to a sharp drop in drivers’ PR times. Highway access is the third important factor which would result in lower PR times (<3 s). Among the protective roadside marking and infrastructure, warning pier is the most effective way to guide drivers. The existence of corrugated-steel guardrails has a negative impact on PR times over 1 s, so that they would accelerate the PR process. Concrete guardrails and traffic markings/signs have similar low influence on PR times. Each additional opposite vehicle results in additive negative effects. This means that a high volume of opposite traffic increases driving task difficulty and decreases PR times. All the distance of curve appearance parameters are positive, and they have larger impacts on longer PR time. This conforms to common knowledge. However, curve distance between 30 m–50 m has a greater positive influence on both short and long PR time compared with curve distance over 50 m. This reveals that drivers responded dispersedly in such circumstance. PR times increase with large longitudinal impact force (steep slope) and decrease with vertical impact force (rough road surface). This shows the impact of road geometry on drivers’ PR times. Drivers’ real-time operation consists of speed and acceleration changing rate, but they are not significant in the prediction model.
4. Perception-Response Time Assistance Model
4.1. Model Structure
According to the PR time prediction model and the recommended range of PR times introduced in Section 2.2, this section will build a PR time assistance model to help drivers to respond timely and properly on a mountain highway curve. Figure 7 describes the flowchart of the assistance model.
Figure 7.
Perception-response time assistance model flowchart.
4.2. Model Calibration
There were six tests conducted on each type of curve. Six drivers did one test on six types of curves respectively. The PR time prediction results are shown in Figure 8.
Figure 8.
Results of 6 × 6 test. The numbered bars represent the tests with inappropriate PR time.
In Figure 8, the black bar stands for the tests of which the predicted PR time is inappropriate (beyond or below the recommended range). The dash line is the recommended range of PR time according to the 85th quantile in Figure 4. The PR times of most drivers were within this range. On near-left curves, no sample was beyond the recommended range. All cases (black bars) with unsuitable PR time were detected by the assistance model (see Table 7).
Table 7.
Perception-Response Time Assistance Model Performance.
Among the five curve types, the far-left curves tended to result in a large deviation on PR times, followed by the middle-right curve. This means that when a far-left curve appeared, some drivers would take reaction within 1 s and other drivers tended to respond far more slowly. Average dispersion change of CV had more improvement on right-turning curves than left-turning curves after assistance, especially on near-right curves. This would result from the fact that the right-turning process has more flexibility than left-turning process according to the traffic rule in China. The possibility of an unsafe turning process (large value of three CVs) would increase with different PR times. Hence proper assistance strategy can help drivers to behave well in such circumstances.
5. Conclusions
This study defined the concept of perception-response time (PR time) on mountain highway curves. Field tests were conducted with a driving recorder to record drivers’ real-time driving data both before and during the turning process. As drivers’ visual information is an important factor in perception-response time prediction in this study, driver-vision lane model was used to extract the elements of visual information from the sight of the drivers. To predict the perception-response times, a multinomial log-linear prediction model with the elements of traffic/road environment, driver-vision lane model, and mechanical status was presented. A corresponding assistance model was also illustrated to help drivers to have more proper perception-response times on mountain highway curves.
The results showed that the same road situation could result in various understandings regarding the interpretation of perception-response times in this work. Compared with the length of driver-vision lane, curvature rate (especially ) had more impact on PR times. This indicates that drivers’ PR process is mostly based on the visual information integrality of curves. Uncertainty of the curve shape and curvature rate will lead to uncertain PR times, such as observed with the middle curve (middle-VAR: 1.162, near-VAR: 0.609, far-VAR: 0.826). Meanwhile, the effect of road safety infrastructure are less significant than that of driver-vision lane elements. Therefore, good road visual geometry is more effective than passive road safety infrastructure in driving behavior guidance.
On the other hand, this study verified the need for establishing a personal-targeted assistance model, which took drivers’ personal behavior and visual information into consideration. This is more humanized than a traffic-rule-based system. Although the model in this study has been proven valid, the absolute data and the meaning obtained should be considered only as relative result [29] due to the limitation of low-frequency data. More high-frequency and accurate biological information detectors would contribute to future studies. The change of eye gaze position/area would also help to illustrate drivers’ different perception processes to establish a more personalized assistance system. In the future, complex driving behavior and driver-based decision processes will be analyzed and quantified.
Acknowledgments
The authors acknowledge the support from National Science & Technology Support Program of China (2014BAG01B06) and National Science Foundation of China (51078270, 51238008).
Author Contributions
Yi Li and Yuren Chen conceived and designed the experiments; Yuren Chen performed the experiments; Yi Li analyzed the data; Yi Li and Yuren Chen wrote the paper.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Guizzo, E. How Google’s Self-Driving Car Works. IEEE Spectrum Online. 18 October 2011. Available online: http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-car-works (accessed on 30 August 2016).
- Bubb, H. Fahrerassistenz primär ein Beitrag zum Komfort oder für die Sicherheit? [Assistance systems—A win of comfort or safety?]. VDI Berichte 2003, 1768, 25–44. [Google Scholar]
- Werneke, J.; Kassner, A.; Vollrath, M. An analysis of the requirements of driver assistance systems—When and why does the driver like to have assistance and how can this assistance be designed. In Proceedings of the European Conference on Human Centred Design for Intelligent Transport Systems, Loughborough, UK, 30 June–1 July 2008; pp. 193–204.
- Iglesias, I.; Isasi, L.; Larburu, M.; Martinez, V.; Molinete, B. I2V communication driving assistance system: On-board traffic light assistant. In Proceedings of the IEEE 68th Vehicular Technology Conference, (VTC 2008-Fall), Calgary, AB, Canada, 21–24 September 2008; pp. 1–5.
- Pfeiffer, J.; Strauss, M.; Rück, E.; Stählin, U.; Lüke, S.; Komar, M. Driver Assistance Program. U.S. Patent Application, No. 12/864,747, 30 January 2019. [Google Scholar]
- Ministry of Transport of the People’s Republic of China. Design Specification for Highway Alignment; JTG D20-2006; China Communications Press: Beijing, China, 2019.
- Totzke, I.; Naujoks, F.; Mühlbacher, D. Warnungen vor Stauenden: Nutzen Fahrer präzise Positionsangaben bei Stauendewarnungen? [Traffic jam ahead warning: Do drivers need information of the position of the traffic jam?]. In Intelligente Transport-und Verkehrssysteme und-dienste Niedersachsen eV (Ed.), AAET—Automatisierungssysteme, Assistenzsysteme und eingebettete Systeme für Transportmittel; ITS Niedersachsen: Braunschweig, Germany, 2011; pp. 79–97. [Google Scholar]
- Olson, P.L.; Sivak, M. Perception-response time to unexpected roadway hazards. Hum. Fact. J. Hum. Fact. Ergon. Soc. 1986, 28, 91–96. [Google Scholar]
- Varhelyi, A.; Adell, E.; Alonso, M.; Plaza, J. HMI Literature Review. Report C20. 52b. PReVENT Consortium. Available online: http://www.google.com.hk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=6&ved=0ahUKEwiXlPS3_YnRAhWGspQKHVSuAesQFgg7MAU&url=http%3A%2F%2Fwww.mj.niedersachsen.de%2Fdownload%2F11065&usg=AFQjCNGLVuECdsjBv77Zmz4yXDGlPLjm5A&cad=rja (accessed on 30 August 2016).
- Scialfa, C.T.; Borkenhagen, D.; Lyon, J.; Deschênes, M.; Horswill, M.; Wetton, M. The effects of driving experience on responses to a static hazard perception test. Accid. Anal. Prev. 2012, 45, 547–553. [Google Scholar] [CrossRef] [PubMed]
- Borowsky, A.; Oron-Gilad, T. Exploring the effects of driving experience on hazard awareness and risk perception via real-time hazard identification, hazard classification, and rating tasks. Accid. Anal. Prev. 2013, 59, 548–565. [Google Scholar] [CrossRef] [PubMed]
- Konstantopoulos, P.; Chapman, P.; Crundall, D. Driver’s visual attention as a function of driving experience and visibility. Using a driving simulator to explore drivers’ eye movements in day, night and rain driving. Accid. Anal. Prev. 2010, 42, 827–834. [Google Scholar] [PubMed]
- Rudin-Brown, C.M.; Edquist, J.; Lenné, M.G. Effects of driving experience and sensation-seeking on drivers’ adaptation to road environment complexity. Saf. Sci. 2014, 62, 121–129. [Google Scholar] [CrossRef]
- Sagberg, F.; Bjørnskau, T. Hazard perception and driving experience among novice drivers. Accid. Anal. Prev. 2006, 38, 407–414. [Google Scholar] [CrossRef] [PubMed]
- Chen, S.W.; Fang, C.Y.; Tien, C.T. Driving behaviour modelling system based on graph construction. Transp. Res. C Emerg. Technol. 2013, 26, 314–330. [Google Scholar] [CrossRef]
- Summala, H.; Lamble, D.; Laakso, M. Driving experience and perception of the lead car’s braking when looking at in-car targets. Accid. Anal. Prev. 1998, 30, 401–407. [Google Scholar] [CrossRef]
- Fuller, R. A conceptualization of driving behaviour as threat avoidance. Ergonomics 1984, 27, 1139–1155. [Google Scholar] [CrossRef] [PubMed]
- Dozza, M. What factors influence drivers’ response time for evasive maneuvers in real traffic? Accid. Anal. Prev. 2013, 58, 299–308. [Google Scholar] [CrossRef] [PubMed]
- Bhattacharjee, A.A.; Wankhede, S.S. Modeling of driver behaviour recognition and prediction using dynamic bayesian network. Int. J. Eng. Res. Gen. Sci. 2015, 3, 691–694. [Google Scholar]
- Lethaus, F.; Harris, R.M.; Baumann, M.R.; Köster, F.; Lemmer, K. Windows of driver gaze data: How early and how much for robust predictions of driver intent? In International Conference on Adaptive and Natural Computing Algorithms; Springer: Berlin/Heidelberg, Germany, 2013; pp. 446–455. [Google Scholar]
- Oliver, N.; Pentland, A.P. Graphical models for driver behavior recognition in a smartcar. In Proceedings of the IEEE Intelligent Vehicles Symposium, (IV 2000), Dearborn, MI, USA, 3–5 October 2000; pp. 7–12.
- Traffic Management Bureau of the Public Security Ministry. Traffic Accident Annual Report of P.R.C. (2014); Traffic Management Bureau of the Public Security Ministry: Beijing, China, 2015.
- Varela, F.J.; Thompson, E.; Rosch, E. The Embodied Mind: Cognitive Science and Human Experience. Am. J. Psychol. 1993, 28, 101. [Google Scholar]
- Fang, S.E.; Guo, Z.Y.; Yang, Z. A New Identification Method for Accident Prone Location on Highway. J. Traffic Transp. Eng. 2001, 1, 91–98. [Google Scholar]
- Tanguy, J.M. Statistical and Semi-Empirical Hydrology. In Physical Processes and Measurement Devices; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2013; pp. 123–165. [Google Scholar]
- Cumulative Frequency Analysis. Available online: https://en.wikipedia.org/wiki/Cumulative_frequency_analysis#Histogram (accessed on 30 August 2016).
- Yu, B.; Chen, Y.; Wang, R.; Dong, Y. Safety reliability evaluation when vehicles turn right from urban major roads onto minor ones based on driver’s visual perception. Accid. Anal. Prev. 2015, 95, 487–494. [Google Scholar] [CrossRef] [PubMed]
- Bland, J.M.; Altman, D.G. Statistics notes. The odds ratio. BMJ 2000, 320, 1468. [Google Scholar] [CrossRef] [PubMed]
- Godley, S.T.; Triggs, T.J.; Fildes, B.N. Driving simulator validation for speed research. Accid. Anal. Prev. 2002, 34, 589–600. [Google Scholar] [CrossRef]
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).