A Testing and Evaluation Method for the Car-Following Models of Automated Vehicles Based on Driving Simulator

Abstract

The continuous advancement of connected and automated driving technologies has garnered considerable public attention regarding the safety and reliability of automated vehicles (AVs). Comprehensive and efficient testing is essential before AVs can be deployed on public roads. Current mainstream testing methods involve high costs in real-world settings and limited immersion in numerical simulations. To address these challenges and facilitate testing in mixed traffic scenarios involving both human-driven vehicles (HDVs) and AVs, we propose a testing and evaluation approach using a driving simulator. Our methodology comprises three fundamental steps. First, we systematically classify scenario elements by drawing insights from the scenario generation logic of the driving simulator. Second, we establish an interactive traffic scenario that allows human drivers to manipulate vehicles within the simulator while AVs execute their decision and planning algorithms. Third, we introduce an evaluation method based on this testing approach, validated through a case study focused on car-following models. The experimental results confirm the efficiency of the simulation-based testing method and demonstrate how car-following efficiency and comfort decline with increased speeds. The proposed approach offers a cost-effective and comprehensive solution for testing, considering human driver behavior, making it a promising method for evaluating AVs in mixed traffic scenarios.

1. Introduction

In recent years, automated vehicles (AVs) have attracted significant global attention due to rapid advancements in information technology [1,2,3,4]. Compared to traditional human-driven vehicles (HDVs), AVs offer potential advantages in addressing various traffic challenges, such as reducing fuel emissions and improving road capacity [5,6]. However, the complexity of the transportation system renders AVs susceptible to safety issues, particularly when operating in mixed environments. As of 15 May 2022, the National Highway Traffic Safety Administration reported 392 crashes involving vehicles equipped with Level 2 advanced driver assistance systems [7], with an additional 11 fatalities confirmed from May to September 2022 in crashes involving automated vehicles [8]. These incidents have heightened public concerns regarding the safety and reliability of AVs. Thus, ensuring that AVs can operate safely, reliably, and efficiently under various scenarios—including high dynamic traffic, adverse weather conditions, and various driving tasks—requires comprehensive, systematic, and rigorous testing before deployment [9,10]. With the increasing deployment of AVs, mixed traffic flows comprising both AVs and HDVs are expected to become commonplace [11]. Consequently, testing AV driving performance in such mixed environments emerges as a critical challenge.

Various testing approaches are employed across different stages of automated driving system development and verification. Huang et al. [12] summarized the integrated method of automated driving testing, including software in the loop (SIL), hardware in the loop [13], vehicles in the loop (VEHIL), and real traffic driving tests. Currently, AV testing primarily relies on two main methodologies: virtual simulation testing and field testing [5]. Virtual simulation employs various software platforms such as PreScan (v8.5.0), PanoSim (v5.0), and CarSim (v2020), each offering distinct advantages in vehicle dynamics, sensor accuracy, and scenario modeling [14,15]. However, its ability to replicate real-world scenarios is limited. In contrast, field testing provides more realistic AV scenarios but is constrained by high costs and limited repeatability, thereby restricting the size of the testing scenario library.

To overcome these limitations, driving simulators equipped with advanced scenario generation logic have emerged as pivotal tools in AV testing methodologies. Originating in the 1930s [16], driving simulators have become foundational in research, enabling detailed exploration of diverse driver behaviors within evolving technological landscapes. These simulators facilitate investigations into the impacts of various technologies and infrastructure elements, ranging from variable information signs [17], in-vehicle systems [18,19], and mobile phone usage [20], to AV integration [21]. With the maturation of autonomous driving technology, driving simulators have significantly expanded, offering a controlled environment that eliminates real-world risks and enables the precise manipulation of variables. Operating within a closed environment enhances the safety and controllability of experimental processes, allowing for precise variable settings and seamless replication of scenarios. This cost-effective approach is underpinned by robust scenario generation logic, ensuring comprehensive integration of software and hardware for scenario testing.

This paper investigates the methodology for testing and evaluating AVs’ car-following models using driving simulators, presenting a novel perspective on AV testing methods and processes. Driving simulators contribute to a more scientific and comprehensive approach by defining scenario elements and generating scenarios. This methodology facilitates the creation of mixed driving environments involving interaction between human-controlled driving simulators and AVs, guided by embedded car-following models. It effectively simulates the behavioral uncertainties of human drivers in response to AV decision and planning models [22].

The contributions of this paper are threefold. Firstly, it proposes a novel testing approach using driving simulators, integrating virtual simulation with real-world testing to ensure precise parameter control and reproducible experimental environments for AV testing. Secondly, it introduces a methodological framework for generating mixed traffic scenarios involving HDVs and AVs, facilitating accurate AV modeling and realistic human driver interactions. Thirdly, it establishes an evaluation framework with comprehensive metrics for assessing AV performance in terms of safety, legal compliance, driving behavior, and comfort.

The remainder of this paper is structured as follows: Section 2 provides an overview of existing work; Section 3 introduces the testing and evaluation methods of AV decision and planning models based on driving simulators; Section 4 details the method verification using a driving simulator (SCANeR^TMstudio and compact driving cockpit CDS-650) to validate the proposed method with car-following behavior examples; Section 5 concludes the paper and discusses the future work.

2. Literature Review

2.1. AV Testing Scenario

The generation of scenarios for AV testing represents a critical first step in the evaluation process. Schieben et al. [23] were pioneers in introducing the concept of “scenario” into the field of AV testing, leading to various interpretations and definitions by other scholars. Elrofai et al. [24] defined the scenario as “the combination of actions and maneuvers of the host vehicle in the passive environment, and the ongoing activities and maneuvers of the immediate surrounding active environment for a certain period”. Several innovative methods have been proposed for scenario generation. Feng et al. [25,26,27,28] introduced a framework that combines augmented reality (AR) testing and testing scenario library generation (TSLG) to assess the safety of highly automated driving systems (ADS) on test tracks. Furthermore, an adaptive testing scenario library generation (ATSLG) method was introduced, offering more efficiency and robustness than TSLG. Fellner et al. [29] applied the rapid search random tree (RRT) method in path planning for scenario generation, resulting in over 2300 scenario elements. Rocklage et al. [30] devised a scenario generation approach based on the backtracking algorithm, enabling random generation of dynamic and static elements within the scenario. Schilling et al. [13] adopted machine learning to alter scenario elements randomly, including changes in white balance, lighting, and motion blur. Gao et al. [31] introduced a test scenario generation method for AVs using a test matrix (TM) and combinatorial testing (CT), along with an improved CT algorithm that balances test efficiency, condition coverage, and scenario complexity, enhancing the complexity of the scheme. Finally, Zhao et al. [32] developed an accelerated test algorithm based on the Monte Carlo method, selectively targeting critical scenarios for random tests, substantially reducing test mileage while improving efficiency. To address the issue of the rarity of safety-critical events, Feng et al. [9] created an adversarial environment for the automated vehicle under test by training the motion of background vehicles. The results showed that the proposed method could accelerate the testing process of AVs and improve efficiency.

2.2. AV Testing Methods

In practical applications of CAV testing, scenario-based testing demonstrates significant advantages in effectively simulating and evaluating the performance of autonomous driving systems in varied and complex environments. It not only aids in revealing system performance under specific conditions but also enhances overall test efficiency and quality, allowing developers to more accurately target potential risks. Therefore, although distance-based testing provides valuable real-world road experience for autonomous driving systems, scenario-based testing has increasingly become the mainstream approach due to its critical role in ensuring overall system safety and adaptability. Based on testing objectives, current research on testing methodologies involves functionality testing, safety testing, compliance and standards testing, and user experience testing. While different testing objectives may share similar testing processes, the underlying scientific questions differ, leading to varying requirements for scenario generation. Based on the execution environment, scenario-based CAV safety testing is primarily divided into simulation testing, closed-track testing, and real-world testing. Additionally, to accelerate the testing process and evaluate CAV safety performance in low-probability, safety-critical scenarios, there is increasing research focused on testing methods specifically targeting corner cases or edge cases.

(1)

Simulation Testing: High-fidelity simulations leverage advanced platforms to create realistic driving environments, enabling safe and controlled testing. These simulations replicate complex interactions and various conditions, allowing developers to assess the CAV’s safety performance and protocols [33,34,35,36]. By integrating Extended Reality (XR), an immersive environment is created [37,38,39], enhancing the understanding of how CAVs perceive and respond to different safety-critical scenarios, which is crucial for identifying potential risks and ensuring robust safety measures.

(2)

Closed-Track Testing: Conducted in controlled environments, closed-track testing mimics real-world conditions, allowing for the safe execution of high-risk scenarios. By systematically running predefined safety scenarios, researchers can observe and measure the CAV’s behavior, response times, and decision-making processes [40,41]. This method ensures precise evaluation of safety performance metrics and highlights areas for improvement before progressing to public road testing, thereby mitigating risks associated with real-world deployment. Some notable closed-track testing facilities around the world include Mcity (Michigan, USA) [42], Millbrook Proving Ground (Bedfordshire, UK) [43], AstaZero (Sweden) [44], and National Intelligent Connected Vehicle Pilot Demonstration Zone (Shanghai, China) [45].

(3)

Real-World Testing: Deploying CAVs on public roads gathers performance data in actual traffic conditions, revealing how well the vehicles handle real-world unpredictability. This phase is crucial for validating the safety of CAVs in dynamic environments. By continuously monitoring and analyzing the CAV’s response to various traffic situations, developers can ensure the vehicle’s ability to manage real-world risks and maintain high safety standards [46,47].

(4)

Corner Case Testing: Focusing on rare but critical scenarios, corner case testing ensures that CAVs can manage unexpected and high-risk situations [48,49]. Stress testing subjects CAVs to extreme conditions, evaluating their robustness and safety under the most challenging circumstances. These tests are essential for ensuring that CAVs are prepared for a wide range of potential hazards, thereby enhancing their reliability and overall safety performance.

As AV testing grows increasingly complex and the potential scenarios become infinitely diverse, simulation testing has gained popularity among researchers and automotive companies due to its efficiency and cost effectiveness. However, field testing remains an indispensable verification step for AVs, bridging the gap between development and market readiness.

2.3. Driving Simulator Applications in AV Testing

In the era of maturing autonomous driving technology, driving simulators’ application has increasingly become prominent. For instance, Zhang et al. [50] designed a human-in-the-loop simulation experiment using a driving simulator to enhance their game-theoretic model predictive controller’s capability in handling mandatory lane changes among multiple vehicles. Human drivers can operate the simulator to exert specific interventions on the test vehicle and further assess the model’s performance. Koppel et al. [51] examined human drivers’ lane-changing behavior using a driving simulator and found significant variability in human lane-changing actions. Suzuki et al. [52] developed an intention inference system that recommends suitable deceleration times to diminish collision risks; the system was rigorously assessed via driving simulator experiments. Mohajer et al. [53] utilized a driving simulator to conduct both subjective and objective evaluations of AVs’ performance, leveraging immersive virtual environments and high-fidelity driving experiences. Manawadu et al. [54] harnessed a driving simulator to analyze individuals’ autonomous driving experiences and the process of manually driving the vehicle.

In conclusion, virtual simulation testing often struggles to create a convincing mixed traffic scenario for AVs, whereas real vehicle testing faces hurdles due to high costs and low repeatability when altering scenarios. Driving simulators, possessing comprehensive scenario generation logic, fill this gap by providing software and hardware support for low-cost testing scenario modeling. Simultaneously, they establish a secure mixed traffic scenario for the testing process. Hence, this paper focuses on studying the testing and evaluation methods of AV planning and decision-making models using a driving simulator.

3. Methodology

3.1. Testing Methods for AVs in Mixed Traffic Scenarios

3.1.1. Framework

Experiments conducted by Kalra and Paddock [55] underscore the necessity for extensive testing of AVs, highlighting the limitations of distance-based approaches. Consequently, the focus of AV testing has shifted towards scenario-based methods [33,56,57,58,59], prioritizing the examination of critical, high-risk situations crucial for evaluating safety and decision-making processes. This approach enables focused and repeated testing of low-probability “long-tail” scenarios and high-risk scenarios, ensuring a comprehensive assessment of autonomous vehicle capabilities across diverse operational conditions. This transition underscores the superior effectiveness of scenario-based testing in simulating diverse driving environments, surpassing mere cost and efficiency considerations. Therefore, the design and generation of scenarios have become the initial steps in the testing process. In the phase of mixed traffic flow, numerous potential traffic safety issues may arise from the unpredictability of HDV operations, the lack of uniform control methods among automated vehicles (AVs) of different levels, and the uncertainties stemming from interactions between HDVs and AVs [60]. Hence, this paper focuses on mixed traffic scenarios, encompassing environments characterized by interactions between AVs and HDVs. Testing in mixed traffic scenarios differs significantly from both traditional vehicle testing and the idealized future testing of fully automated AVs. A comparative summary of these distinctions is outlined in

Table 1. Comparisons of three types of traffic scenarios.

Traffic Scenarios	Traditional Traffic	Mixed Traffic	Ideal Intelligent Traffic
Penetration rate of AVs	0	(0, 100%)	100%
Vehicle automation levels	Consistent (All HDVs)	Inconsistent (Vehicles with different automation levels)	Consistent (All fully autonomous vehicles)
Uniformity of testing criteria	Uniform	Non-uniform	Uniform
Types of vehicle-to-vehicle interaction	HDV-HDV	HDV-HDV AV-AV AV-HDV	AV-AV
Emphasis of testing	Vehicle factory testing	Vehicle factory testing Driving abilities testing Interaction capabilities testing	Vehicle factory testing Driving abilities testing Operational human-friendliness testing

below.

As the intelligence of transportation systems has advanced, human involvement in the driving process has significantly diminished. The roles of human drivers have evolved from active operators to supervisors, often becoming passive observers or completely disengaging from the driving task. Consequently, vehicle-to-vehicle interactions now encompass varying degrees of autonomous driving technology [61]. Traditional interactions limited to human-driven vehicles (HDV-HDV) have given way to interactions involving vehicles with diverse levels of autonomous driving capabilities. Similar to human drivers, who undergo assessments and licensure before operating on public roads, autonomous vehicles, as they assume more driving responsibilities, also require specific assessment criteria.

Therefore, alongside traditional vehicle-centric testing paradigms such as wind tunnel experiments and collision assessments, it is increasingly imperative to integrate testing elements that address human–vehicle interactions. Moreover, unlike the homogeneity and standardization typical of traditional and idealized intelligent traffic scenarios, mixed traffic environments exhibit disparities in automation levels and manufacturing standards, rendering interactions more vulnerable to safety challenges. Consequently, refining testing protocols becomes critical in mixed traffic scenarios. Real-world vehicle testing in mixed traffic settings presents logistical challenges and significant costs, underscoring the need for reliable simulation or X-in-the-loop testing to ensure safety and feasibility.

At the core of this testing paradigm lies the creation of testing scenarios. The approach for generating mixed traffic scenarios in this study draws inspiration from scenario generation processes used in driving simulation software and incorporates the conceptual framework outlined earlier. This architectural framework is illustrated in Figure 1 below.

Figure 1 illustrates the methodology for creating mixed traffic scenarios and conducting autonomous driving tests. Initially, the analysis involves examining the operational procedures of the driving simulator and identifying critical scenario elements. Constraints are then established to tailor these elements for AV testing scenarios. The modeling of testing scenarios entails developing tested vehicle (TV) and background vehicle (BV) models for the scene. Ultimately, the interaction between human-driven and automated vehicles is facilitated through the driving simulator.

3.1.2. Scenario Modeling

Following the framework of scenario elements, testing scenario modeling is conducted through the development of a scene model, a TV model, and a BV model. The details of the scenario modeling are as follows:

• Scene modeling: This involves defining critical road characteristics such as road type, length, starting coordinates, direction, and alignment. Lane specifications include the number of lanes, lane types, permissible vehicle types, lane width, speed limits, and pavement materials. Additionally, the modeling includes surrounding infrastructures such as buildings, integrated traffic signage, street lights, trees, and other static objects. Environmental conditions such as weather, lighting, and air quality are meticulously adjusted to simulate various driving conditions. Collectively, these elements construct a comprehensive and realistic testing environment, facilitating detailed evaluations of automated vehicles’ performance across diverse simulated driving scenarios.
• TV modeling: Modeling the TV is a complex process tailored to meet specific testing scenario requirements. This process entails selecting the TV’s type, brand, model, and color, and positioning it within the designated driving lane at predetermined coordinates. The TV is also subject to speed constraints, typically aligned with the lane’s speed limit. The TV’s planning and decision-making model is structured into a three-tier system: global path planning generates an initial path, behavioral decisions respond to environmental factors, and a trajectory adheres to specific constraints. Software scripting functions then control the TV, ensuring the model accurately reflects real-world driving complexity and responds dynamically to elements introduced in testing scenarios.
• BV modeling: Following the setup of static scenario elements, attention shifts to dynamic scenarios critical for assessing AV driving capabilities. These dynamic scenarios create varied environmental conditions for the TV, ensuring comprehensive testing across diverse contexts. Simulation data from these scenarios provide a holistic evaluation of model performance. BV behavior significantly influences the TV within these dynamic scenarios, with BVs controlled by human drivers using the simulator, introducing realistic driving variations in judgment and behavior.

3.1.3. Testing Method of AV Decision and Planning Model

The testing method for the AV planning and decision-making model utilizes generated scenarios within a driving simulator to create a realistic and immersive virtual driving environment. Experimenters interact with this environment through the simulator’s controls, embedding the AV planning and decision-making model for testing in mixed traffic scenarios. This setup involves two vehicle types: the BV and the TV. The BV is operated by a human driver using a compact driving cockpit equipped with realistic controls such as a steering wheel, transmission, and switches (Figure 2a). This setup enables real-time interaction with the virtual environment, including speed and directional adjustments.

Conversely, the TV is virtually represented in SCANeR Studio and controlled through scripting functions. The model implementation employs a tree-view structure comprising Rules, Conditions, Actions, and Functions (Figure 2b), enabling dynamic and customizable scenarios incorporating road characteristics, vehicle positions, and weather conditions. Detailed scripting within SCANeR Studio enhances control over the TV, ensuring precise evaluation of AV behavior and responsiveness under complex driving conditions.

3.2. Evaluation Method

The evaluation method consists of two components: evaluation indicators and evaluation standards. Evaluation indicators assess the performance of AVs, while evaluation standards define the acceptable ranges for these indicators and are used to measure the testing outcomes.

3.2.1. Evaluation Indicator

This paper categorizes evaluation indicators into two types: qualitative and quantitative. According to the evaluation standards, if all qualitative indicators meet the required thresholds, scores from both qualitative and quantitative indicators are combined to determine the total evaluation score. Conversely, if any qualitative indicator fails to meet these thresholds, the overall testing outcome is deemed unsatisfactory.

• Qualitative indicators. Qualitative indicators evaluate the safety of the TV, including its compliance with regulations and rules, and its adherence to the behavioral attributes expected of the test item.

• Safety indicator Q.

During the testing, the position coordinates (x_n, y_n) of the center point of TV and the position coordinates (x_n−1, y_n−1) of the center point of BV need to be extracted, then the real-time distance D₁ between the two vehicles can be calculated as follows:

$$D_1=\sqrt{{(x_n-x_{n-1})}^2+{(y_n-y_{n-1})}^2}-{\displaystyle \frac{l_n+l_{n-1}}{2}}$$

(1)

where l_n denotes the length of the testing vehicle body, set to 4.95 m, and l_n−1 denotes the length of the testing vehicle ahead, set to 2.07 m.

Furthermore, the theoretical safety distance Gap_safe, and the safety indicator Q also can be calculated as follows:

$$Q=D_1-Ga{p}_{safe}$$

(2)

• Distance D₂ from the contour of the vehicle to the edge of the roadway.

During the testing, the position coordinates (x_n, y_n) of the center point of TV and the ordinate y_b of the edge of the roadway are extracted, and then, the distance from the contour of the vehicle to the edge of the roadway can be calculated as follows:

$$D_2=y_n-y_b-{\displaystyle \frac{w_n}{2}}$$

(3)

where w_n denotes the width of the TV.

• Distance D₃ from the vehicle contour to the roadway boundary line.

During the testing, the position coordinates (x_n, y_n) of the TV center point and the ordinate y_f of the roadway boundary line are extracted, and then the distance from the vehicle contour to the roadway edge line can be calculated as follows:

$$D_3=y_f-y_n-{\displaystyle \frac{w_n}{2}}$$

(4)

• Behavior attribute indicator G.

It is determined according to the expected behavior of TV in the testing; generally, it is the difference in actual value and theoretical safety value.

$$G=X_0-X_{safe}$$

(5)

where X₀ denotes the actual value of the behavior attribute of TV, and X_safe denotes the theoretical safety value of the behavior attribute.

2.

Quantitative indicators. Quantitative indicators are used to evaluate the comfort level of TV and the completion efficiency of the expected behavior of the tested item.

• Expected behavior completion efficiency u.

It can be achieved through the actual value of the behavior attribute indicator divided by its maximum range as follows:

$$u={\displaystyle \frac{G}{G_{\max }-G_{\min }}}$$

(6)

where G_max and G_min, respectively, denote maximum and minimum boundary values within the allowable range of behavioral attribute indicators.

• Comfort indicator j.

In an autonomous driving environment, the comfort indicator is a key factor in evaluating the driving ability of an AV. The comfort indicator can be judged by the maximum acceleration in the lateral and longitudinal directions.

$$j_x={\displaystyle \frac{\widetilde{a_x}}{{\tilde{a}}_x^{\max }}}$$

(7)

$$j_y={\displaystyle \frac{\widetilde{a_y}}{{\tilde{a}}_y^{\max }}}$$

(8)

where j_x denotes the comfort indicator of the tested vehicle in the longitudinal direction, j_y denotes the comfort indicator of the tested vehicle in the lateral direction, a_x denotes the maximum longitudinal acceleration of the tested vehicle, set to 3 m/s², ${\tilde{a}}_x^{\max }$ denotes the maximum allowable longitudinal acceleration in comfortable, a_y denotes the maximum lateral acceleration of the tested vehicle, set to 5 m/s², and ${\tilde{a}}_y^{\max }$ denotes the maximum lateral acceleration allowed in comfortable.

3.2.2. Evaluation Criteria

• Safety evaluation. The actual distance between TV and BV must be no closer than the theoretical safe distance, namely,

  $$Q>0$$

  (9)
• Violation evaluation. The outline of the vehicle must not cross the edge of the roadway and the boundary line of the roadway, namely,

  $$D_2>0,D_3>0$$

  (10)
• Behavioral attribute evaluation. Determine the range of behavioral attribute indicators based on experimental conditions. The range is used as the benchmark of calculation efficiency, and the real result in the testing should be in the range.

$$G_1<G<G_2$$

(11)

where G₁ denotes the minimum boundary value of the expected behavior attribute, and G₂ denotes the maximum boundary value of the expected behavior attribute.

4.

Efficiency evaluation. Set a total score of w₁. According to the proportional relationship, the corresponding scores R₁ can be achieved as follows:

$$R_1=(1-u)\times w_1$$

(12)

5.

Comfort evaluation. The total score can be set to w₂. The lateral and longitudinal comfort indicator scores each account for half. According to the proportional relationship, the longitudinal comfort score value R_x, the lateral comfort score value R_y, and the final score R₂ can be obtained.

$$R_x=(1-j_x)\times {\displaystyle \frac{w_2}{2}}$$

(13)

$$R_x=(1-j_x)\times {\displaystyle \frac{w_2}{2}}$$

(14)

$$R_2=R_x+R_y$$

(15)

4. Method Verification

4.1. Model under Testing

The car-following model is a theoretical framework that employs dynamic methods to analyze the behavior of vehicles traveling in a single lane without the ability to overtake. This section verifies the proposed method using three classic car-following models: the GM model, the Gipps model, and the Fuzzy Reasoning model.

• GM model. Developed in the late 1950s, the GM model is characterized by its simplicity and clear physical interpretation [62]. This pioneering model, significant in the early research of car-following dynamics, derives from driving dynamics models. It introduces a concept wherein the “reaction”—expressed as the acceleration or deceleration of the following vehicle—is the product of “sensitivity” and “stimulation”. Here, “stimulation” is defined by the relative speed between the following and leading vehicles, while “sensitivity” varies based on the model’s application consistency. The general formula is outlined below,

  $$a_{n+1}\left(t+T\right)=c{v}_{n+1}^m\left(t+T\right){\displaystyle \frac{\Delta v\left(t\right)}{\Delta x^l\left(t\right)}}$$

  (16)

  where a_n+1(t + T) denotes the acceleration of the vehicle n + 1 at t + T, v_n+1(t + T) denotes the speed of the vehicle n + 1 at t + T, ∆v(t) denotes the speed difference between the vehicle n and the vehicle n + 1 at t, ∆x(t) denotes the distance between the vehicle n and the vehicle n + 1 at t, and c, m, l are constants.

It is assumed that acceleration is proportional to the speed difference between two vehicles, and inversely proportional to the distance between the heads of the two vehicles. And at the same time, it is directly affected by its own speed.

2.

Gipps model. Originating from the safety distance model, the Gipps car-following model [63] utilizes classical Newton’s laws of motion to determine a specific following distance, diverging from the stimulation-response relationship used in the GM model. This model computes the safe speed for the TV relative to the vehicle ahead (BV) by imposing limitations on driver and vehicle performance. These limitations guide the TV in selecting a driving speed that ensures it can safely stop without rear-ending the preceding vehicle in case of a sudden halt. The mathematical expression of this model is outlined below,

$$v_n(t+\tau )=\min \left\{\begin{array}{l}{v}_n(t)+2.5a_n\tau (1-{\displaystyle \frac{v_n(t)}{V_n}})\sqrt{0.025+{\displaystyle \frac{v_n(t)}{V_n}}}\\ -b_n({\displaystyle \frac{\tau }{2}}+\theta )+\sqrt{b_n^2{({\displaystyle \frac{\tau }{2}}+\theta )}^2+b_n\left\{2\left[x_{n-1}(t)-x_n(t)-S_{n-1}\right]-\tau v_n(t)+{\displaystyle \frac{v_{n-1}^2(t)}{b_{n-1}}}\right\}}\end{array}\right.$$

(17)

where v_n(t) denotes the speed of vehicle n at t, a_n denotes the maximum acceleration of vehicle n, V_n denotes the speed limit of the vehicle n in the current design scenario, b_n denotes the maximum deceleration of the vehicle n, b_n−1 denotes the maximum deceleration of the vehicle n − 1, x_n(t) denotes the position of vehicle n at t, S_n−1 denotes the effective length of the vehicle n − 1, τ denotes the reaction time of the vehicle n − 1, θ denotes the additional reaction time of the vehicle n − 1 to ensure safety.

3.

Fuzzy Reasoning model. Initially proposed by [64], the Fuzzy Reasoning model focuses on analyzing driving behaviors by predicting the driver’s future logical reasoning. This model fundamentally explores the relationship between stimulus and response, similar to other behavioral models. A distinctive characteristic of the Fuzzy Reasoning model is its use of fuzzy sets to manage inputs. These fuzzy sets, which partially overlap, categorize the degree of membership for each input variable. The mathematical formulation of the model is presented below,

If ∆x is appropriate, then,

$$a_{n,i}=(\Delta v_i+a_{n-1,i}\times T)/\gamma$$

(18)

where ∆x denotes the distance difference between the two vehicles, a_n,i denotes the acceleration of vehicle n at time i, a_n−1,i denotes the acceleration of vehicle n−1 at time i, ∆v_i denotes the speed difference between the two vehicles, T denotes the reaction time, which is 1 s, and TV expects to catch up with the preceding vehicle within time γ; here, it is 2.5 s.

If ∆x is not appropriate, a_i will decrease 0.3 m/s² as ∆x decreases by one level, and it will increase 0.3 m/s² as ∆x increases by one level.

$$a_{n,i}={\displaystyle \frac{\Delta v_i+a_{n-1,i}\times T}{\gamma }}+0.3\Delta \mathrm{x}$$

(19)

4.2. Mixed Traffic Scenario Construction

This paper utilizes SCANeR Studio and the compact driving cockpit CDS-650 as experimental platforms. SCANeR Studio is a comprehensive driving simulation software package renowned for its ergonomic interface, suitable for simulating a wide range of vehicles from light to heavy. It is extensively used in vehicle ergonomics, advanced engineering studies, road traffic research and development, as well as human factor studies and driver training.

SCANeR Studio integrates robust scenario generation logic essential for AV testing, enhancing the operational efficiency required for scenario creation and simulation experiments. The operational process within SCANeR Studio for scenario generation and simulation experiments comprises five modules: Terrain, Vehicle, Scenario, Simulation, and Analysis.

Scene model. A scene model is designed based on the characteristics of the car-following model and the specific requirements of AV testing. To enhance the realism and immersion of mixed driving testing scenarios, the roadside landscape includes office buildings, schools, and additional structures, interspersed with green trees and bushes. Weather conditions are set to sunny, with parameters such as snow, rain, clouds, and fog all set to 0%, and light set to 80%. The scenario elements are detailed in

Table 2. Scenario elements’ setup in scenario modeling.

Basic Road Information		Lane Information		Environmental Information
Road type	Urban expressway	Number of lanes	Two-way two-lane	Weather	Sunny 80%
Road length	1000 m	Lane type	Asphalt concrete		Snow, rain, clouds, and fog at 0%
Starting point	Origin	Type of vehicle	car	Lighting	80%
Road direction	0°	Lane width	3.75 m	Landscape	Office buildings, schools;
Road alignment	Straight line	Lane speed limit	120 km/h		Green trees and bushes

.

The schematic diagram of the scenario model is shown in Figure 3.

• TV model. The vehicle selected as the TV in SCANeR Studio is the “Citroen_C3_Green”. The TV is positioned at the starting point of the road. Speed limits used in this study are set at 60 km/h, 90 km/h, and 120 km/h. Custom control of the TV is implemented through scripting based on the car-following model mentioned earlier.
• BV model. The vehicle designated as the background vehicle (BV) in SCANeR Studio is the “Citroen_C3_Red”. The BV is placed 20 m in front of the TV and is controlled by a human driver using the driving simulator. Prior to the experiment, the following procedures are conducted: (1) Ensure the proper functioning of the driving simulator equipment. (2) Set the speed limit of TV. (3) Align the BV’s speed limit with that of the TV. (4) Start the BV and gradually accelerate to the speed limit according to driving habits. During this process, the BV can be accelerated and decelerated irregularly. (5) Towards the end of the road (approximately 50 m from the endpoint), initiate emergency braking maneuvers to decelerate the BV’s speed to zero. (6) Terminate the simulation, and the testing ends. The total travel distance covered by the BV throughout the experiment is approximately 980 m. Prior to formally commencing the experiment, preliminary trials were conducted with each participant’s session duration limited to 30 min. The successful completion of these preliminary trials indicated that the experiment duration was generally within the tolerable range for participants. This precaution aimed to mitigate potential discomfort or dizziness that participants might experience during the experiment.

4.3. Mixed Traffic Scenario Construction

• Safety evaluation. The theoretical safety distance of Gap_safe is as follows:

  $$Ga{p}_{safe}=v_1\left(t\right){\tau }_1+{\displaystyle \frac{v_1{\left(t\right)}^2}{2b_1}}-{\displaystyle \frac{v_0{\left(t\right)}^2}{2b_0}}$$

  (20)

  where, Gap_safe denotes the safety distance of TV, b₀ denotes the maximum deceleration of BV, b₁ denotes the maximum deceleration of TV, τ₁ denotes the reaction time of the driver of TV, v₀(t) denotes the speed of BV at time t, and v₁(t) denotes the traveling speed of TV at time t.

In the safety evaluation, the score is set to 15. As in Formula (9), when the actual distance between TV and BV is not less than the theoretical safe distance, the safety performance is qualified.

2.

Violation evaluation. The total score for this section is set at 30 points. As in Formula (10), the contour of the vehicle does not cross the edge of the roadway and the boundary line of the roadway, the evaluation is qualified. If D₂ > 0, 15 would be obtained. Similarly, if D₃ > 0, 15 would be obtained.

3.

Car-following characteristic evaluation. In the car-following project testing, the car-following characteristic indicator G is expressed by the difference between D₁ and the theoretical safety distance Gap_safe.

$$G=D_1-Ga{p}_{safe}$$

(21)

In the evaluation of car-following characteristics, the total score is set to 15. When the real driving track of the tested vehicle is within a reasonable range, it is determined that it meets the following characteristics: the left boundary of the range is the trajectory of the safe driving distance, and the right boundary is the trajectory of the safe driving distance translated by 0.2 times of the theoretical safety distance, namely,

$$0<G<0.2Ga{p}_{safe}$$

(22)

4.

Car-following efficiency evaluation. The car-following efficiency indicator u is expressed by the difference between D₁ and Gap_safe divided by 0.2 times Gap_safe.

$$u^t={\displaystyle \frac{{D_1}^t-Ga{p_{safe}}^t}{0.2Ga{p_{safe}}^t}}$$

(23)

where u^t denotes the efficiency of sampling point t during the driving process, $D_1^t$ denotes the actual distance between the two vehicles at point t, $Ga{p}_{safe}^t$ denotes the theoretical safe distance at point t.

$$u={\displaystyle \frac{u^1+u^2+\dots +u^t\dots +u^n}{n}}$$

(24)

where u denotes the average efficiency, and n is the number of sampling points.

In the car-following efficiency evaluation, a total score of 20 points is set. According to the proportional relationship, the score value R₁ corresponding to different values of efficiency can be obtained.

$$R_1=(1-u)\times 20$$

(25)

5.

Comfort evaluation. A total score of 20 points is set, of which the horizontal and vertical comfort indicator scores each account for 10 points. According to the proportional relationship, the longitudinal comfort score R_x, the lateral comfort score R_y, and the final score R₂ can be obtained.

$$R_x=(1-j_x)\times 10$$

(26)

$$R_y=(1-j_y)\times 10$$

(27)

$$R_2=R_x+R_y$$

(28)

According to the above evaluation indicators and evaluation criteria, the results of the car-following ability evaluation are shown in

Table 3. Evaluation results of car-following ability.

Evaluation Items	Requirements	Judging Results
Safety	Q ≥ 0	15 points/Unqualified
Car-following characteristics	0 < G < 0.2 Gapsafe	15 points/Unqualified
Violation	D2 > 0	15 points/Unqualified
	D3 > 0	15 points/Unqualified
Car-following efficiency	R1 = (1 − u) × 20	R1
Comfort	R2 = Rx + Ry = (2 − jx − jy) × 10	R2
Total score	Qualified: R = 60 + R1 + R2	R
	Unqualified: R = 15 × r

Note: r denotes the number of qualified qualitative indicators; car-following ability test is passed only if three qualitative evaluation items are qualified; otherwise, it fails; when the test result is qualified, the quantitative evaluation item score is valid, the total evaluation score can be calculated, Otherwise, it is unqualified and the corresponding score is less than 60.

below.

4.4. Results

The experimental design aims to evaluate the car-following model of the TV at three different speed limits: 60 km/h, 90 km/h, and 120 km/h. Each comprehensive test group comprises nine experiments, involving three distinct models and the three specified speed limits. This setup results in a total of 54 test groups, with six different drivers conducting each group test.

Participants aged from 22 to 25, with 0 to 3 years of driving experience, and a balanced gender distribution, operate the simulator based on their individual driving habits and abilities. The experiment does not strictly categorize participants by gender, age, or driving experience.

In the setup, the BV is designated as vehicle No. 0, and the TV as vehicle No. 1. Figure 4a displays the overall scenario effect with the inclusion of both TV and BV, while Figure 4b offers a view from the driver’s perspective of the TV during the tests.

The experimental results from six groups of experiments conducted by six participants have been compiled, showing the total evaluation scores (R) for the Gipps model, GM model, and Fuzzy Reasoning model under speed limits of 60 km/h, 90 km/h, and 120 km/h, as presented in

Table 4. Experimental results.

Group 1	60 km/h	90 km/h	120 km/h	Group 2	60 km/h	90 km/h	120 km/h
Gipps Model	71.4308	77.6676	45	Gipps Model	74.5781	76.7838	66.5414
GM Model	45	45	45	GM Model	45	45	45
Fuzzy Reasoning Model	45	45	45	Fuzzy Reasoning Model	45	45	45
Group 3	60 km/h	90 km/h	120 km/h	Group 4	60 km/h	90 km/h	120 km/h
Gipps Model	77.3397	71.1287	72.8375	Gipps Model	73.8108	65.7953	63.3177
GM Model	45	45	45	GM Model	45	45	45
Fuzzy Reasoning Model	45	45	45	Fuzzy Reasoning Model	45	45	45
Group 5	60 km/h	90 km/h	120 km/h	Group 6	60 km/h	90 km/h	120 km/h
Gipps Model	70.6911	68.9145	65.1808	Gipps Model	76.5556	69.3031	67.7908
GM Model	45	45	45	GM Model	45	45	45
Fuzzy Reasoning Model	45	45	45	Fuzzy Reasoning Model	45	45	45

.

From the data in Table 4, further analysis is warranted. Average scores for the Gipps model are 74.0677, 71.5985, and 67.1336 under speed limits of 60 km/h, 90 km/h, and 120 km/h, respectively. The experimental results indicate that neither the GM model nor the Fuzzy Reasoning model met the qualifying criteria, particularly falling short on safety indicators. Consequently, in cases where qualitative benchmarks are not met, scores for their car-following efficiency and comfort metrics are not computed. Hence, the scores for the GM model and Fuzzy Reasoning model are derived from the sum of scores for car- following characteristics (15 points) and violation (two sub-items, 15 × 2 = 30 points), totaling 45 points. Given that the safety evaluation criteria employed in this study are derived from the safety-distance model, specifically rooted in the Gipps model, which demonstrates superior safety performance compared to the GM and Fuzzy Reasoning models, these findings further affirm the efficacy of the evaluation methodology utilized. Given that the safety evaluation criteria employed in this study are derived from the safety-distance model, specifically rooted in the Gipps model, which demonstrates superior safety performance compared to the GM and Fuzzy Reasoning models, these findings further corroborate the efficacy of the evaluation methodology utilized.

Furthermore, in order to study the relationship between car-following efficiency, comfort and driving speed, the average scores of the six groups of experiments of R₁ and R₂ based on the Gipps model at 60 km/h, 90 km/h and 120 km/h speed limits, and data results are shown in

Table 5. Average score of Gipps model R₁ and R₂.

	60 km/h	90 km/h	120 km/h
R1	9.2454	8.4706	3.5456
R2	4.8223	3.1283	2.3991

.

From Table 5, the average scores of R₁ in the six experiments under the speed limit of 60 km/h, 90 km/h, and 120 km/h based on the Gipps model are 9.2454, 8.4706, and 3.5456, respectively. Correspondingly, the average scores of R₂ are 4.8223, 3.1283, and 2.3991, respectively. It can be seen that the car-following efficiency and comfort level of the Gipps model decrease with the increment in driving speed.

The speed change curves for the BV, the TV, and the car-following efficiency score (R₁) during the experiment are depicted in Figure 5. The blue dotted line represents the BV’s speed curve, the blue solid line represents the TV’s speed curve, and the red line represents the score of the car-following efficiency indicator. If speed decreases, R₁ increases sharply, which further verifies the inverse relationship between the car-following efficiency and the driving speed.

4.5. Limitations and Outlook

This study has focused on relatively simple and fundamental models for both model selection and scenario construction, which may have constrained the richness and generalizability of the results. The choice of classic models and the restriction to interactions between only two vehicles in platooning scenarios limit the complexity and diversity of simulated driving behaviors. Future research endeavors will need to address these limitations by exploring more complex and varied scenarios, as well as refining model selection and calibration processes. In particular, advancing research should emphasize the study of intricate driving environments that better reflect real-world complexities. This includes scenarios with heterogeneous traffic compositions and varying levels of vehicle autonomy, aiming to capture a broader spectrum of driving behaviors and interactions.

Moreover, in practical applications, it is essential for practitioners and policymakers to consider the implications of road speed limits and the driving speeds of AVs. This aspect is crucial as it directly impacts traffic efficiency, safety, and user acceptance. Future studies should thus prioritize detailed investigations into optimal speed profiles for AVs to ensure safe, smooth, and efficient integration into mixed traffic environments.

5. Conclusions

Testing AVs is crucial to ensure their safe and efficient operation across various road traffic scenarios. Initially, field tests used real vehicles, but virtual testing has gained traction due to its safety, repeatability, and cost-effectiveness. However, both methods face challenges in creating realistic mixed traffic scenarios. To address these limitations, we propose a new testing method using a driving simulator. In this method, a human driver manipulates the simulator’s cockpit components to interact with the AV’s planning and decision-making model in SCANeR Studio, effectively simulating mixed traffic scenarios with human driver randomness.

Experiments employing three classic car-following models—the GM model, the Fuzzy Reasoning model, and the Gipps model—yielded significant findings. Firstly, safety evaluation criteria favor the Gipps model, with experimental results confirming its superior performance over the GM and Fuzzy Reasoning models, consistent with theoretical expectations. Thus, the outcomes derived from the driving simulator are considered valid. Secondly, the car-following efficiency and comfort level of the Gipps model decline as driving speeds increase. Future research could focus on the secondary development of driving simulator software to allow custom control of tested vehicles and the integration of sensors and other software and hardware components for comprehensive AV testing.

Overall, the proposed method using the driving simulator proves to be a reasonable and effective approach for assessing AVs’ planning and decision-making models in mixed traffic scenarios, thereby enhancing the safety and efficiency of interactions between HDVs and AVs.