Evaluation of Autonomous Vehicle Takeover Performance in Work-Zone Environment ^†

Abstract

The advent of autonomous vehicles (AV) could revolutionize the automotive industry by significantly improving safety, efficiency, and accessibility. Despite their potential to improve traffic safety by reducing human error, their integration into existing transportation systems presents significant challenges. This is particularly evident in scenarios involving takeover events, where there is a transition of control from the vehicle to the human driver. Our driving simulator study, involving 14 drivers in a work-zone environment, provides critical insights into the takeover performance of level 3 to level 5 AVs. The findings suggest that the successful integration of AVs depends on their seamless incorporation into existing systems and the readiness of drivers to adapt to this emerging technology.

1. Introduction

Autonomous Vehicles (AVs) represent a transformative advancement in transportation technology, garnering significant attention for their potential to redefine travel by enhancing safety, efficiency, and accessibility. As AV technology progresses, concerns and questions arise, particularly regarding takeover events—instances where there is a transition of control from the vehicle to the human driver [1].

Despite the promise of AVs in enhancing traffic safety by reducing crashes caused by human error, there are still considerable challenges, particularly in mixed-traffic environments. Reports from the Department of Motor Vehicles in California, USA, indicate that the primary reasons for AV disengagements are as follows: (1) discrepancies in road maps; (2) proximity to lanes or road boundaries; and (3) misinterpretation of the surrounding environment [2]. Recognizing these limitations is crucial, especially in complex environments like work zones. As car automation advances, public acceptance of AVs has increased significantly. It was found that acceptance levels for AVs at levels 0 and 4 increased, with Level 4 experiencing the most significant rise, of 1.37 times [3,4]. Despite increased acceptance, Level 3 AVs still face reduced acceptance due to concerns about the need for the transition of control from the vehicle to the human driver in the event of unexpected situations causing the AV to disengage. AV technology aims to reduce injuries, enhance mobility, and relieve drivers of the responsibilities of driving [5]. However, in Level 3 automation, drivers must promptly recover control when the vehicle reaches its functional limit. This situation poses a challenge, as drivers, no longer actively monitoring the environment, may experience a decline in situational awareness, making it difficult to assume control during a takeover request (TOR) [6].

Takeover performance, defined as a driver’s capacity to regain control in autonomous driving situations, is critical for integrating AVs into transportation networks [7]. This process has significant safety implications, influencing preferences, perceived comfort, acceptance, and trust in automation. Various studies have designed diverse takeover scenarios to examine their impact on takeover performance. For example, the effect of TOR modality (visual vs. visual plus auditory) was explored in the following three scenarios: missing lane markings; temporary lanes due to a work zone; and high road curvature [8]. It was found that drivers exhibited better lateral control with visual–auditory TORs, especially in high-curvature scenarios. Other studies expanded on this by introducing variations in automation levels and non-driving-related tasks (NDRTs) [9]. It was observed that NDRTs increased perceived situation criticality in work-zone scenarios. High automation levels led to increased variability in the lateral vehicle position and perceived situation criticality in high-curvature scenarios.

Recovering control of an AV in predictable situations, such as work zones, poses significant challenges. It has been suggested that high-predictability scenarios, like those with warning signs before work zones, impact driver awareness and responses [10]. Conversely, it was also found that initiating the takeover process reduces both trust in AVs and takeover performance [11]. Familiarity with TOR can improve takeover performance and trust calibration during initial system use if participants are briefed on TOR warnings before simulations [12]. The TOR, typically a dashboard pop-up and/or warning sound, is crucial in human–machine interactions. Some research has indicated that visual warnings for takeovers significantly enhance control handover, while another study found that drivers recognize and respond to TORs more effectively when visual and auditory warnings are combined [13,14].

Freeway and highway reconstruction or maintenance often elevate the risk of traffic crashes. The literature identifies several temporary traffic-control countermeasures designed to mitigate these increased risks in work zones [15]. Work zones disrupt drivers’ expectations of road geometry, necessitating sudden speed adjustments [16]. Real-time prediction and monitoring challenges impede timely traffic-control implementation by traffic management, contributing significantly to traffic accidents. For example, approaching fog zones requires drivers to take control of AVs. Different traffic-flow conditions influence AV drivers’ behaviors, highlighting the need to analyze Level 3 AV drivers’ behaviors after regaining control [17]. Addressing AV operations in work zones has been challenging for researchers and car manufacturers due to the dynamic nature and varied layouts. Frequent changes in traffic-control plans complicate coding for each situation. Key issues for AV movement in work zones include identifying the work zone, recognizing lane closures, navigating alternative routes, interpreting traffic signs, and interacting with other drivers [18].

The primary objective of this research is to investigate the takeover performance among adults utilizing Level 3 and Level 5 autonomous vehicles in a simulated work-zone environment. The study focuses on the relationship between cognitive abilities, mental workload, situational awareness, and the effectiveness of takeover maneuvers.

2. Experimental Method

This research examines takeover performance differences among 14 individuals aged between 22 and 44 using a driving simulator. The study aims to understand how mental workloads vary between AV automation Levels 3 and 5, and their impact on driver performance during a takeover in a work-zone environment. The experimental design is based on a selected layout from the Hungarian road design guidelines for work zones [19]. The first sign is 1000 m before the lane closure and diversion point (Figure 1). This is where the takeover request is activated. The driver has 1 km to complete the takeover process before entering the work-zone section, known as the critical takeover distance. The simulation incorporates road markings and traffic signs in highway work zones to accurately assess drivers’ takeover performance in a controlled environment that closely emulates real-world scenarios.

2.1. Apparatus

BeamNG.drive software was used to provide a controlled environment that was beneficial for realistic driver simulations [20]. The simulator setup includes an Intel workstation PC with RTX 3060 GPU and three 32-inch curved monitors, providing a 180-degree visual experience. Eye-tracking was used for measurements and to provide insights into visual attention, which is crucial for observing, understanding, and anticipating the ongoing takeover process [21]. We used Pupil Labs Neon wearable eye-tracking glasses equipped with a mobile companion to capture the raw data. Post-processing was conducted using Pupil Neon Player, version 4.1.2 and exported for further Python and MS Excel analysis.

2.2. Procedure

The driving experiment consisted of three scenarios. Before the experiment, participants were briefed on the scenarios, including the TOR and its appearance. To ensure familiarity with the simulator, all participants, despite their varied backgrounds, underwent a tutorial and drove in the simulator for five minutes on a highway similar to the experimental scenario.

Scenario A—Full Autonomous Driving (Level 5): Participants experienced completely autonomous driving. The vehicle autonomously performed all driving tasks including accelerating, braking, and steering. The primary aim was to measure mental workload levels during full autonomy.

Scenario B—Human-Driven Mode: Participants engaged in manual driving, allowing for the assessment of performance metrics and mental workload under manual control.

Scenario C—Autonomous Driving with Takeover Request (Level 3): The third scenario involved Level 3 autonomous driving. It began with automated driving and transitioned into a work zone, where a TOR was initiated (an audio-visual sign appeared), prompting participants to take manual control. This scenario aimed to measure takeover performance metrics.

2.3. Takeover Performance Measurements

Driver takeover in conditionally automated vehicles is influenced by several factors, as follows: driver characteristics (age, gender); NDRTs (cognitive load, emotional states); vehicle configuration (TOR lead time, TOR modality); and driving environment (traffic density, weather). This study evaluates the impact of these variables on takeover transitions by analyzing driving behaviors following the transition to manual control (TOR) compared to conventional and higher-level automation scenarios. Driving behaviors were categorized into takeover timeliness and takeover quality to assess takeover performance. Takeover timeliness was measured by the response time from TOR to the initial takeover maneuver. Takeover quality encompasses a range of metrics, including speed, acceleration, jerk statistics, time/distance to collision statistics, steering angle and pedal statistics, lane deviation statistics, and crash rate [6]. The present study employed a set of takeover performance indices, including response capability and quality metrics.

3. Results

The metrics under consideration included speed, steering intensity (steering angle range, the difference between a vehicle’s maximum and minimum steering angles), throttle, brake, hands on steering wheel (0–2), heart rate, mean duration of fixation, total entries of fixation, and blink frequency (

Table 1. Mean values and statistical metrics of measured data.

		Speed (km/h)	Steering Intensity (Degree)	Throttle (%)	Brake (%)	Hands on Steering Wheel	Heart Rate (bpm)	Total Entries of Fixation	Duration of Fixation [ms]	Blinks Frequency (Blinks/min)
A	Mean	71.20	40.74	0.17	0.03	0.20	77.82	172.33	762.65	26.36
	SD	7.29	20.94	0.03	0.02	0.46	13.19	39.59	156.18	12.18
B	Mean	81.00	39.22	0.18	0.01	1.71	84.68	170.13	762.63	20.66
	SD	14.20	26.29	0.04	0.01	0.21	13.40	46.23	274.12	12.01
C	Mean	82.60	46.90	0.20	0.01	1.33	79.51	160.93	933.42	23.22
	SD	13.93	41.48	0.05	0.01	0.40	10.01	67.28	477.17	15.11

).

The Wilcoxon signed-rank test was employed to compare the distributions of each metric between these conditions. The significance threshold was set at a p-value of 0.05. Additionally, Spearman correlation coefficients were computed to evaluate the linear relationships between the metrics under each pair of conditions.

The results of the Wilcoxon signed-rank test are presented below:

• Speed (km/h): The Wilcoxon test revealed significant differences in speed between conditions A and B (p = 0.0479) and conditions A and C (p = 0.0054). These results suggest that the driving speed varied significantly under B and C scenarios, indicating changes in driver behavior compared to the fully automated driving scenario (A);
• Brake (%): Significant differences were observed in brake usage between conditions A and B (p = 0.0054) and conditions A and C (p = 0.0067). This indicates that braking patterns were affected by the driving conditions. Thus, manual driving greatly affects brake usage compared to autonomous driving;
• Hands on steering wheel: This metric showed significant differences across all comparisons (A–B, p < 0.0001; A–C, p = 0.0015; B–C, p = 0.0015), suggesting that the amount of time drivers kept their hands on the steering wheel varied significantly between the scenarios;
• Heart Rate (bpm): This metric exhibited significant differences between conditions A and B (p = 0.0008) and B and C (p = 0.0020). The consistent differences in heart rate indicate varying levels of stress or exertion experienced by the drivers under different scenarios;
• Blink Frequency (blinks/min): There were significant differences in blink frequency between conditions A and B (p = 0.0215) and A and C (p = 0.3894), indicating changes in driver alertness or fatigue levels;
• The scenarios did not significantly affect the throttle and fixation values.

The Spearman correlation insights are as follows:

• Heart Rate presented strong significant correlations across all pairs: A and B (ρ = 0.95, p < 0.001), A and C (ρ = 0.94, p < 0.001), and B and C (ρ = 0.88, p < 0.001). This suggests that heart rate is a reliable indicator of changes in driver states;
• Blink Frequency also showed significant positive correlations (A–B: ρ = 0.70, p < 0.001; A–C: ρ = 0.81, p < 0.001; B–C: ρ = 0.92, p < 0.001), indicating its potential use as a measure of driver alertness.

4. Discussion

Significant differences in metrics, such as speed, brake usage, hands on steering wheel, heart rate, and blink frequency, highlight the impact of driving conditions on driver behavior and physiological state. The analysis of the data reveals several notable results:

• Speed increased from A to B and A to C, indicating improved driver confidence and a cautious autonomous driving style. An increased difference between B and C is not expected. Still, drivers accelerated to their speed after taking control and were driving more quickly because of the “uncertainty” of pedal control at the start;
• Brake percentage reductions from A to B and A to C imply more efficient manual driving or greater precaution in autonomous driving, with an increase from B to C, indicating more frequent braking during the takeover;
• Heart rates increased from A to B and A to C, indicating higher stress or excitement, while a decrease from B to C suggests reduced stress or more relaxed conditions if autonomous driving was engaged. The mean heart rate was highest in scenario B due to the stress of manual driving. In scenario C, the heart rate increased after the driver took over and then remained constant;
• Blink frequency reductions from A to B and A to C imply increased concentration, whereas an increase from B to C suggests a return to more normal, relaxed patterns.

5. Conclusions

The study underscores the considerable influence of driving conditions on driver behavior and physiological responses. When developing autonomous systems, it is imperative to consider driver engagement and stress levels. The higher heart rates and variable blink frequencies observed in manual driving suggest increased stress and concentration. This indicates that automated systems should seek to achieve a balance between driver control and system assistance to reduce stress and enhance safety.

The correlation between heart rate and other driving metrics indicates that physiological measures can indicate the driver’s state. Changes in brake usage, heart rate, and blink frequency provide insights into the driver’s state and conditions, which can aid in further analysis of driving performance and safety factors. Future research should explore takeover performance in more complex driving scenarios and the possibility of real-time physiological monitoring in driving systems.