Modeling Driver’s Real-Time Confidence in Autonomous Vehicles

Abstract

Autonomous vehicle technology has developed at an unprecedented rate in recent years. An increasing number of vehicles are equipped with different levels of driving assist systems to reduce the human driver’s burden. However, because of the conservative design of its programming framework, there is still a large gap between the performance of current autonomous driving systems and experienced veteran drivers. This gap can cause drivers to distrust decisions or behaviors made by autonomous vehicles, thus affecting the effectiveness of drivers’ use of auto-driving systems. To further estimate the expected acceptance of autonomous driving systems in real human–machine co-driving situations, a characterization model of driver confidence has to be constructed. This paper conducts a survey of driver confidence in riding autonomous vehicles. Based on the analysis of results, the paper proposes a confidence quantification model called “the Virtual Confidence (VC)” by quantifying three main factors affecting driver confidence in autonomous vehicles, including (1) the intrusive movements of surrounding traffic participants, (2) the abnormal behavior of the ego vehicle, and (3) the complexity of the driving environment. The model culminates in a dynamic confidence bar with values ranging from 0 to 100 to represent the levels of confidence. The validation of the confidence model was verified by doing comparisons between the real-time output of the VC and the real-time feeling of human drivers on an autonomous vehicle simulator. The proposed VC model can potentially identify features that need improvement for auto-driving systems in unmanned tests and provide data reference.

1. Introduction

The technology related to autonomous vehicles (AVs) has developed rapidly in recent years. Researchers are engaged in developing more advanced and intelligent autonomous driving systems [1,2]. With the increased deployment of autonomous driving systems in ordinary vehicles, it has become a major research goal to make the systems perform more like human drivers to maintain safety and stability in unexpected events [3].

However, even human drivers may lose confidence in their driving ability when they encounter unexpected or potentially dangerous situations on the road. Studies on human drivers have found that a number of drivers will feel a lack of confidence when performing specific actions in certain scenarios [4,5], such as overtaking, highway insertion, and traffic jams. Furthermore, many of them also face a lack of confidence in some first-encounter scenarios. Studies [6,7] have demonstrated that driving with low confidence can result in a greater incidence of incorrect response actions. Even without serious consequences, drivers who lack confidence while driving can have a detrimental effect on traffic flow and contribute to higher accident rates [8]. For autonomous driving, there is also a need for human drivers to have confidence in the performance of autonomous vehicles during human–machine interaction. The lack of confidence in autonomous driving mainly stems from uncertainty about the vehicle’s operational behavior, such as whether it will make a safe lane change when overtaking or avoid interference from oncoming traffic on side roads or highways. Additionally, uncertainty arises from the consequences of unexpected events that have not been encountered in previous autonomous driving experiences.

1.1. The Limitations of Autonomous Vehicles

There are several reasons why autonomous driving systems are not yet widely used. (1) the current urban infrastructure is not fully equipped to support certain types of autonomous driving systems. For example, due to cost constraints, there are relatively few cities with infrastructure capable of supporting the Internet of Vehicles (IoV) technology [9]; (2) the perfection of autonomous driving systems is still insufficient to achieve a lower accident rate under fully automated operation. In 2021 alone, the California auto driving database recorded 260 accidents involving autonomous vehicles [10]; (3) the average consumer has a low level of trust in autonomous driving systems and believes that their own driving is more efficient than auto driving [11].

Consumer distrust is the most influential factor limiting the promotion of autonomous driving. There are a few models available for purchase that claim to have some autonomous driving capability, such as Tesla’s FSD-equipped models, Nio’s NAD-equipped models, and the Mercedes-Benz S-Class. However, most autonomous driving models are optional configurations that can be quite expensive due to the high cost of the hardware equipment required [12]. When consumers cannot trust the autopilot system to function properly, they may only be willing to spend money out of curiosity. However, the autonomous driving system needs to be a standard feature that is as reliable and affordable as Advanced Driving Assistance Systems (ADAS) [13] that are embedded in the price of the vehicle. Due to the high cost of autonomous driving, the stress and lack of confidence experienced by drivers when using it become significant obstacles to its promotion.

1.2. The Reasons for Distrusting Auto Driving

Those who have experienced an autonomous driving system will understand the sense of frustration it can bring to drivers during its operation. Current autonomous driving systems have significant deficiencies in both their intelligence or human-like nature, as well as the human–computer experience during their operation [14]. Some imperfect autonomous driving systems do not issue any warning signals in scenarios where there is no potential for a direct collision, such as when there is pedestrian activity on the side road or through a traffic-light-free intersection. This is because the perception system determines that there is no direct interaction threat, and the vehicle decides not to slow down as a normal driver would. This can result in a high driver takeover rate. Additionally, current autonomous vehicles lack the ability to inform the driver of the reasons for their decisions [15]. Only the results of the decisions are presented on the interface, which limits the experience of human–computer interaction. This leads to drivers lacking confidence in the autonomous driving system in certain situations and choosing to take over the control. This paper argues that the problem of human–machine exchange can be solved through testing and optimization. However, the driver’s lack of confidence due to the dangerous behavior of the autonomous driving system is a relatively difficult problem to define. Some operations that do not seem to be problematic from the perspective of the vehicle, such as changing lanes in a safe situation, can still affect the driver’s confidence if they are performed multiple times in a short period. Additionally, when the autonomous driving system’s control is unstable, it can not only lead to dangerous consequences but can also interfere with the driver’s judgment.

According to the mentioned influence of driver confidence on autonomous driving systems, we believe that it is necessary to conduct a survey on the current level of confidence in driving autonomous vehicles and to summarize the main factors influencing confidence. Based on the relationship between vehicle driving and those influencing factors, some numerical formulas should be developed to form a virtual driver’s real-time confidence model. The confidence model is expected to provide a reference for the occupant state during unmanned testing of AVs, identify unreasonable boundary scenarios, and assist in optimizing the system for non-safety-related issues, thereby enabling the widespread use of autonomous driving.

In this paper, we conducted a social survey on the factors that significantly affect driving confidence when using auto-driving systems, summarized several important reasons, and investigated the degree of people’s willingness to take over. The results of the survey were analyzed in detail, and the VC model is constructed based on three aspects: external vehicle interference, ego vehicle actions, and environment complexity. The validity of the virtual confidence model was finally verified through the volunteers’ marking process. The paper is structured as follows: Section 2 presents the results of the survey on the main factors that affect confidence when using autonomous driving systems. In Section 3, we demonstrate the VC model and provide detailed calculations of its components. Section 4 analyzes the test results of the VC model. The future of the VC model is discussed in Section 5, and Section 6 concludes the paper.

2. Survey on Main Factors Affecting Confidence

The survey of confidence in the use of autonomous driving systems was conducted among random members of the community who have had experience with autonomous driving systems. The first round of the survey was a small open-ended question asking “What are the main reasons an autonomous driving system makes you feel less confident while driving? ”. 15 interviewees responded to the question. The results were collated to generate a total of 15 confidence confounders, as follows:

Confidence Confounders

• AVs sudden acceleration, braking or turning
• Traffic participants’ approaching
• Surrounding vehicles’ sudden braking or acceleration
• Approaching scenarios like cross-road or tunnel
• Numerous buses or large trucks nearby
• Numerous pedestrian nearby
• Existence of blind zone
• Bad weather condition
• High velocity
• High-frequency lane change
• Poor brands with low prices
• Short system up-time
• Low volume production scale
• Unable to visualize perceptual or path info
• Small numbers or types of sensors

Research has shown that a driver’s driving characteristics can affect their driving confidence [16]. However, in order to facilitate the identification of more universally applicable factors influencing driver confidence, we did not take into account the driving characteristics of the drivers in our survey.

Based on the main influencing factors, we constructed a large-scale study of drivers’ confidence in their willingness to take over. 180 copies of the study were sent out and 135 were returned.

Respondents were asked to give their confidence level for each factor, ranging from Level 1 to Level 5, with lower levels representing lower confidence and a higher probability of taking over. Level 1 indicates 0 scores, and Level 5 indicates 5 scores. The results of the study are shown in Figure 1. The confidence score in the figure is obtained by calculating the average.

It can be seen from the figure that the proximity of other traffic participants to the ego vehicle has the greatest impact on driver confidence, with a mean confidence score of only $2.36$. The second most influential factor is the sudden acceleration and deceleration or sharp turning and the frequent lane change of the ego, with a mean confidence score of $2.47$. Other factors that have a greater influence are the high velocity of the ego, the large number of pedestrians nearby, and the sudden acceleration or braking of surrounding vehicles. Among the numerous factors, the behavior of the AV itself brings the most obvious confidence value reduction, followed by the influence of some surrounding participants or the environment. Furthermore, some inherent factors, such as automatic driving system manufacturers, vehicle brands, etc. will also have an impact on driving confidence but are lower compared to other reasons, for which confidence scores are above $3.3$.

Based on the results of our research, we identified three factors that affect the real-time confidence of human drivers in autonomous vehicles: (1) external traffic participant behavior, which is dominated by surrounding vehicles that may interfere with the ego vehicle; (2) the behavior of the ego vehicle, which is dominated by abnormal acceleration and deceleration; and (3) the complexity of the surrounding environment, which is composed of numerous complex attributes.

To construct the relationships between the various types of influences, a reasonable framework is required. Driver confidence is a quantity that is continuously influenced, whereas the occurrence of events is usually defined by moments. Therefore, the overall trend of confidence values should be a long-term cumulative value that is driven by the influence values of key moments. However, according to the results of the survey, the three major influences on confidence are all negative. If they are modeled as lowering confidence, the longer the autonomous driving system functions, the lower the confidence must be. According to the analysis of human driving experience, when a driver encounters an unexpected event, their driving confidence is likely to be affected to some degree. However, if the impact is not significant enough to make the driver give up driving, the confidence will gradually be restored and built up as the vehicle continues to be driven.

We divided the three main factors into two direct negative factors and one indirect influence. The direct impact includes the ego abnormal movement and external interaction that will directly lead to a decline in confidence. The indirect influence is the environmental complexity that will change the speed of confidence recovery. The architecture of driver confidence can be briefly expressed as Figure 2.

3. Modeling Confidence

We propose a real-time quantification model called “the Virtual Confidence (VC)” for simulating a driver’s driving confidence during the AV operating process. The VC can emulate the real-time acceptance pattern of human drivers about the status of the vehicle by considering three main components: (1) Quantification of the integrated threat to the AV (2) Quantification of the AV’s driving status and violations (3) Quantification of the complexity of the environment in which the AV is exposed. In this section, the design of the specific components and the preliminary algorithm scheme is described and analyzed in detail.

3.1. Model Framework

The calculation of the confidence value can be expressed by the following equation. The operating logic of the VC model can be understood by combining the content of Figure 2 with Equation (1).

$${Conf}_{t+1}=\left\{\begin{array}{c}{Conf}_t-{Of}_t-{Sf}_t+{Re}_t\hfill \\ Mf,\mathrm{if}{Conf}_{t+1}>Mf\hfill \\ 0,\mathrm{if}{Conf}_{t+1}<0\hfill \end{array}\right.$$

(1)

In the equation, ($Conf$, $Of$, $Sf$, $Re$, $Mf$) refers to (Virtual confidence value, Outside influencing value, Self influencing value, Recovery value, Maximum virtual confidence value). Where t and $t+1$ indicate two adjacent moments in terms of frequency. $Conf$ is the specific confidence value at the corresponding moment. In the operation process, this confidence value is controlled to be no more than a maximum value of $Mf$ and no less than 0. The confidence value is specified to be the maximum value at $t=0$ when the vehicle system starts. We specify the $Mf$ as 100 when the surrounding complexity is in the most stable state. $O{f}_t$ for the vehicle’s out factors indicates that the vehicle at the time of t is affected by the external potential threat. This value is the sum of the influence value of the diverse disturbance that the vehicle is subjected to, including the influence of other vehicles’ lane changing, the influence of vehicles’ trajectory overlapping, etc. $S{f}_t$ is the self-factor of the ego vehicle, which indicates the sum value of the abnormal behavior’s impact or unusual state of the vehicle at the moment t, including continuous violation of regulations or interference with other traffic participants. $R{e}_t$ is the amount of confidence value’s recovery. The value of $R{e}_t$ is obtained by calculating the complexity of the driving environment around the ego and converting it using the entropy method. The calculation details of the factors in Equation (1) will be fully described in the following sections. Section 3.2 is for $O{f}_t$, Section 3.3 is for $S{f}_t$ and Section 3.4 is for $R{e}_t$ and $Mf$.

The output values of the VC model are presented in the form shown in Figure 3.

It should be mentioned that the dynamic influence on a human driver’s confidence usually exists within the observable range of the driver. In our confidence model, we have set a fixed computational constraint range for all calculations of influence factors, which means only the factors within the range of the ego vehicle will be considered.

3.2. The Influence of Out Factors

In the process of driving a vehicle, the interference of human drivers comes mainly from the surrounding traffic participants, especially moving vehicles [17]. Therefore, we focus on direct external influences on nearby vehicles that may intersect spatio-temporally with the ego vehicle.

Human drivers can subconsciously predict the movement of surrounding objects and even the possible thoughts and behaviors of surrounding drivers. Once the predicted results present a certain possibility of danger, driving confidence decreases, and response actions are made synchronously. At the same time, humans are able to predict not only the occurrence of a collision but the approximate location of the potential accident, including the location of the road where the collision may occur and the damaged position of the vehicle. To simplify the calculation cost, we adopt the assumption of the “Safety Force Field” system proposed by NVIDIA, which assumes that traffic participants on the road are rational humans who will take brake response when danger is predicted in any situation [18]. Based on this assumption we set up a simple Spatio-temporal intersection prediction.

The prediction can determine the magnitude of the disturbance to the ego vehicle based on the interval. Assuming the emergency response measure performed by the rational person is simple linear braking, then the expected distance traveled by each vehicle can be described as the following equation:

$$d=vs.t_0+\frac{v^2}{2a}$$

(2)

where v represents the instantaneous speed of the vehicle. $t_0$ represents the average braking reaction time. a is the deceleration of emergency braking. For autonomous vehicles, precise calculations of the vehicle’s speed, acceleration, and other related information can be obtained through onboard devices such as GPS, IMU, and cameras [19,20]. For human drivers, the prediction of the moving target will not only focus on the position of the vehicle’s gravity center but also on the moving position of the whole physical entity of the vehicle in space. We assume that the vehicle reachable area is a rectangle extending the length of expected travel distance d from the left and right corners of the vehicle to its forward direction based on the brake assumption. Each active vehicle, on the road around the ego, generates a predicted reachable area, and the intersection of two or more reachable areas can generate different kinds of polygonal intersection areas. The regions intersecting with the ego vehicle are quadrilateral in most cases and triangular or pentagonal in a few, as shown in Figure 4.

In Figure 4, (a) and (c) are the general interacting scenarios among two vehicles where the interaction regions are all quadrilateral, and (b) and (d) are where the regions are triangular or pentagonal appear in the specific situation. In the calculation process, we use the two-dimensional coordinates of the vertices of each vehicle’s reachable regions to calculate the intersection set and obtain the coordinates of the interaction polygon. The properties of interactions are regular under the simple definition, so the coordinates of the vertices can relatively easily be figured out.

After obtaining the vertex coordinates, the positions of the closest and farthest points to the ego and the target vehicle can be obtained by comparing the distances to each vertex, respectively.

The relationship between distance and time to possible collision is expressed as Equation (3):

$$d_i=\left\{\begin{array}{cc}vt,\hfill & \mathrm{if}{d}_i\le vs.t_0\hfill \\ v{t}_0+\frac{a{t}^2}{2},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(3)

We can use the equation to calculate the time required for the front of two vehicles to the nearest point of the interaction area and the two vehicles’ rear to the farthest interaction point. When $d_i\le vs.t_0$, solve for t is the required time. When $d_i>vs.t_0$, the required time is $t+t_0$. This calculation is aiming to figure out the timestamp when the interacting vehicle is expected to drive into and out of the interaction area. By using Equation (4), we can obtain four time points, the entry time $t_{e1}$ and the exit time $t_{e2}$ of the ego vehicle and the entry time $t_{p1}$ and the exit time $t_{p2}$ of the corresponding vehicle:

$${(t_1,t_2)}_i={(t_{e1},t_{e2})}_i\cap {(t_{p1},t_{p2})}_i$$

(4)

The intersection of the two time periods $(t_1,t_2)$ is calculated. i denotes the vehicle serial number that has an intersection with the reachable area of the ego, and if this intersection is empty, it means that under the assumption of this paper, there is no possibility of a future collision between the two vehicles. If the intersection set $(t_1,t_2$) exists, then calculate the collision possibility of ego using Equation (5):

$$P_i=\frac{{(t_1,t_2)}_i}{{(t_{e1},t_{e2})}_i}$$

(5)

where $P_i$ denotes the temporal crossover rate between the ego and the vehicle with serial number i during the interactions.

In addition to the degree of temporal crossover of the two vehicles, the location of the interaction region relative to the ego vehicle can lead to different degrees of influence on driving confidence. We use the minimum distance ${de}_i$ from the front of the ego vehicle to each interaction area, combined with the temporal crossover rate above, to calculate the final influence of interaction on confidence. Using Equation (6), the value of the interference during interaction can be figured out.

$${Inter}_i=k{e}^{(\gamma -{de}_i)}(-ln(1-P_i))$$

(6)

where k is a human-set parameter, which indicates the different impact coefficients of different types of vehicles. For example, the k for large delivery vehicles is higher than that for sedans. $\gamma$ is a human-set parameter, which determines the standard value of interaction loss at the reference distance.

In addition to the interaction interference, the confidence of human drivers will also be affected by the abnormal actions of the surrounding traffic participants. The sudden braking operation of the surrounding vehicles may represent a sudden dangerous situation. In the calculation, we only focus on the vehicle with the maximum acceleration around the ego. The influence of the vehicle on the confidence value can be expressed as Equation (7):

$$E=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}a<\overline{a}\hfill \\ e^{k(1-e^{-(a-\overline{a})})},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(7)

where k is the human-set parameter. a is the acceleration value of the maximum accelerating vehicle. $\overline{a}$ is the constant value for abnormal acceleration, which can be regarded as the response threshold and can always be set to be 7 m/s${}^2$. m is the human-set constant used to determine the minimum amount of affection in case of abnormal acceleration. As the abnormal acceleration’s value rises, the confidence impact will present an exponential expansion.

By combining the interaction and the abnormal behavior confidence interference, the combined effect of two major external interference factors on the ego can be described as Equation (8):

$${Of}_t={(k_1\sum {Inter}_i+k_{2}E)}_t$$

(8)

where t in the equation represents the time of the calculation. $k_1$ and $k_2$ indicate the proportional relationship between the two main influences.

3.3. The Influence of Self Factors

We define the relationship related to the time of the autonomous driving system publication as the baseline coefficients for the self-factor value before calculating the confidence risk associated with the abnormal behavior and state of the ego vehicle, as the following Equation (9):

$$U=e^{\frac{1}{k_{0}y+k_1}}$$

(9)

where y is the length of time the system has been in use. $k_0$ is a set parameter, used to adjust the relationship between driving confidence and the length of time the system has been released. $k_1$ is also a set parameter, used to determine the initial value for the new autonomous driving system.

From the kinetic aspect to detect the abnormal state of the ego, we believe that as long as the ego vehicle is in a state of especially high acceleration or deceleration, as well as a high wheel steering rate can lead to the loss of confidence. We adopt the same equation as the abnormal acceleration and deceleration of the other car in Equation (7) with the different threshold settings to guarantee comfort. We use the rate of change of the ego vehicle’s heading angle to represent the wheel steering which is shown in Equation (10):

$$T=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}\Delta {yaw}_t<\frac{n}{f}\hfill \\ e^{k(1-e^{-(f\Delta {yaw}_t-n)})}+m,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(10)

where f is the operating frequency of the VC model. $\Delta {yaw}_t$ is the value of yaw change at the moment t. n is the baseline changing rate. k is the human-set parameter and m is the constant that is used to determine the minimum amount of loss.

Combining with the abnormal state, the final value of confidence loss due to the unusual driving state of the ego can be expressed as Equation (11):

$${State}_t={(k_1{E}_{ego}+k_2{T}_{ego})}_t$$

(11)

where $E_{ego}$ is the value of abnormal confidence loss for acceleration and deceleration, $T_{ego}$ is the value of confidence loss for abnormal steering rate. t represents the moment t as mentioned above. $k_1$ and $k_2$ are set parameters that can adjust the numerical proportional relationship between the two affections.

For the possible violations of autonomous vehicles, we take into account three common but relatively not completely unacceptable violations that can bring direct danger to the vehicle: (1) Driving over the speed limit (2) Driving against traffic (3) Frequent Lane changes.

For speeding, the traffic law allows vehicles to exceed a certain percentage of the road limit for a short period of time during overtaking. However, when the vehicle is seriously speeding, the greater margin of speeding the greater the negative impact on drivers’ confidence. At the same time, the instantaneous speeding penalty should be capped. We use the following Equation (12) to quantify the impact of the ego vehicle’s speeding:

$$B_v=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}v<v_{lim}\hfill \\ \frac{k}{1+e^{-(v-v_{lim}(1+c_{tol}))}},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(12)

where $v_{lim}$ is the speed limit of the road section. $c_{tol}$ is the set over speeding amplitude. k is a set parameter, which determines the maximum value of the impact of each time period because of the speeding reason.

In the case of going against traffic, although overtaking on a two-lane road will require temporally driving on the opposite lane, the duration of driving on the opposite lane should not be too long. There will be an amount of impact on the driver’s confidence when traveling in the reverse lane. The VC model will check the vehicle’s heading and the road heading. While the two heading value is significantly different, the confidence loss calculation will be carried out according to Equation (13).

$$B_d=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}{yaw}_{ego}\subseteq ({yaw}_{road}-\frac{\pi }{2},{yaw}_{road}+\frac{\pi }{2})\hfill \\ \mathrm{\Lambda },\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(13)

The loss constant $\lambda$ in Equation (13) can be set artificially to describe the driver’s tolerance of avoiding or penalizing the vehicles against traffic time.

The behavior of continuous fast lane change is relatively easy to define, it only needs to set a fixed loss of confidence for each lane change, which can generate a large number of penalty values in a short period of frequent lane change scenarios. Once the ego vehicle touches the road edge, it can be considered a lane change.

$$B_c=\left\{\begin{array}{cc}\mathrm{\Theta },\hfill & \mathrm{if}t=line \& t-1\ne line\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(14)

$\mathrm{\Theta }$ in Equation (14) is the loss constant that can be set artificially. The value of $\mathrm{\Theta }$ should not be set too large to avoid excessive confidence impact caused by the usual lane change.

The equation for the confidence loss of the ego vehicle’s law-breaking actions is as follows:

$${Lord}_t={(k_1{B}_v+k_2{B}_d+k_3{B}_c)}_t$$

(15)

The overall self-factors influence on confidence value can be synthesized as Equation (16).

$${Sf}_t=U(k_1{State}_t+k_2{Lord}_t)$$

(16)

where $k_1$ and $k_2$ are the set parameters. U for the publication time according to Equation (9).

3.4. The Recover Rate

The value of recovery in the driving confidence model is negatively correlated with the complexity of the dynamic and static surrounding environment in which the vehicle is driven [21,22]. This paper specifies that ${Re}_t$ in Equation (1) is always a value not less than 0.

We refer to the design framework of safety entropy [23] for the solution of environmental complexity and determine the influencing factors in safety entropy by investigating the environmental factors. The factors that affect the environmental complexity are divided into three parts: (1) The traffic participant’s properties and status, (2) The static environment of the current road where the ego vehicle is traveling, and (3) The real-time weather condition. The specific details of the influencing factors can be seen in

Table 1. Influencing factors.

Road Traffic Participants	Road Static Environment	Weather Conditions
(1) Average speed	(1) Radius of the current road	(1) Rainfall
(2) Maximum speed difference	(2) Distance from the boundary	(2) Visibility
(3) The number of surrounding participants	(3) Road types	(3) Travel time
(4) Number of large vehicles	(4) Number of lanes	(4) Wind speed

.

As can be seen from Table 1, each of the three major factors affecting environmental complexity is subdivided into four complexity-related subfactors.

We normalize the values of the subfactors in Table 1 and unifies them as the greater the value the higher the impact. In addition, all the values are positive, and the normalization equation is as shown:

$$x_{norm}=1.001-e^{-x}$$

(17)

where x represents the value of the influence subfactor. The constant 1.001 is set to avoid a normalized value of 0.

We analyzed the survey report and combined the results with real-world driving experience to figure out that human drivers will drive relatively more cautiously in scenarios where the average speed of the surrounding active traffic participants is higher [24]. In the road traffic participants’ part, higher average speeds increase the danger level in case of an accident. Meanwhile, the larger the maximum speed difference between the surrounding active state participants, the greater the diversity in their behavior, making cautious driving necessary in such scenarios. Greater numbers of surrounding traffic participants and large vehicles such as trucks and vans require a higher level of caution. The closer the road is to the border, the greater the influence on driver confidence.

We consider the type of road construction, such as viaducts or underground passages, when assessing the influence on driver confidence. We assign a fixed value to this factor to deal with the non-quantifiable problem, for example, 0 on a common road and 1 underground. For the number of lanes factor, the more the number of lanes in the same direction of the current road, the more operational options for vehicles, and the higher the complexity.

For the weather condition factors, we adopt some simple quantification methods for factors: Rainfall is quantified as mm/h. The visibility parameter is quantified as m, and the visibility should take inverse preprocessing. Furthermore, the wind speed parameter’s unit m/s. The travel time factor is non-quantifiable which includes day and night labels.

The overall calculation of complexity is as follows. Equation (18) for each main part to solve the occupancy rate of its indicators.

$$P_i=\frac{x_i}{{\sum }_{i=1}^{i=4}{x}_i}$$

(18)

The information entropy $S_j$ for each major type was obtained using Equation (19). Furthermore, the weighting relationship between each major category of influencing factors was obtained using Equation (20).

$$S_j=\sum _{i=1}^{i=4}-P_{i}ln{P}_i$$

(19)

$$W_j=\frac{1-S_j}{{\sum }_{j=1}^{j=3}(1-S_j)}$$

(20)

Finally, we use the specific normalized values of each subfactor to calculate the entropy value of each major factor category by using Equation (21). Combining the weights in Equation (20), we can obtain the overall complexity value of the AVs’ driving environment named $Co$.

$${En}_j=\sum _{i=1}^{i=4}-x_{i}ln{x}_i$$

(21)

$$C=\sum _{j=1}^{j=3}{W}_j{En}_j$$

(22)

$$Co=\mu C-\rho$$

(23)

In Equation (23), $\mu$ is the numerical amplification and $\rho$ is the standard environmental norm. The cooperation of the two quantities serves to amplify and obtain a more pronounced change in the entropy value. By setting reasonable parameters, the distribution of environmental complexity values obtained by our experimental calibration and their corresponding complexity levels are listed in

Table 2. Environment complexity status.

Level	Value	Env Status
1	(0, 0.21]	Most stable
2	(0.21, 0.24]	Stable
3	(0.24, 0.28]	Median
4	(0.28, 0.33]	Unstable
5	(0.33, $+\infty$)	Most Unstable

:

Based on the value of the environmental complexity, the amount of recovery of real-time driving confidence can be set. A simulation program with a clock interval of 0.05 s is used in this paper, and according to the size of this interval, we first divide five different environmental complexity levels. Where each level will correspond to a number to facilitate the quantification of the recovery amount. The recovery value can be calculated as Equation (24).

$${Re}_t=k\frac{{Conf}_t}{100S_{co}}$$

(24)

$S_{co}$ in the equation corresponds to the value of the complexity level. ${Conf}_t$ is the current vehicle’s confidence value. We assume that the recovery of confidence value should be relatively difficult when confidence is already in a lower value state. This results in the higher the complexity of the environment, the longer time it takes to re-establish confidence. k is the base recovery amount constant.

The maximum value of confidence $Mf$ can also be set according to the $S_{co}$. However, in this paper, we specify $Mf$ as 100 in order to simplify the verification process.

4. Test and Evaluation

We constructed two typical complex scenarios using the 51SIMONE 3.1.1 simulation system and utilized Matlab 2021 to develop a real-time calculation model for VC values to verify the validity of the proposed system. The first scenario focuses on the interaction between the surrounding traffic participants and the ego vehicle, and the second scenario is the abnormal action of the ego vehicle. During the simulation test, the virtual confidence of the AV before entering the scenario is without damage, and the confidence values through the scenario will be recorded simultaneously.

We observe the trend of the confidence value output of the VC model to indicate the reasonableness and effectiveness of the algorithm. At the same time, 15 volunteer testers are asked to ride in the simulator which contains a screen in front of the seat displaying the view of the virtual driver from 51SIMONE. The testers cannot operate the vehicle. After the simulation scenarios are completed, we will request volunteers to evaluate the real-time VC values calculated by our model. The evaluation score will range from 0 to 10, with higher scores indicating greater accuracy of the VC model assessment.

4.1. Scenario One

The background of scenario one is a single 4-lane road. There are four vehicles including the ego and one pedestrian on the simulation road. At the beginning of the scenario, the sidecar on the left lane of the ego vehicle quickly changes its direction to the same lane as the ego. This action lasts for a second, then the left-side vehicle slows down and cancels the lane change. During this period, the driving confidence value of the ego drop-down continuously from 100 to 81.47. Furthermore, for a while, all the vehicles in the scenario keep the current state, which makes the ego confidence recover for a bit. Then, the ego vehicle starts acceleration to catch up with the front vehicle in the same lane. The speed of the ego temporarily exceeds the road’s speed limit, and meanwhile, the left-side car suddenly accelerates to catch up with the ego. These two events occur in synchrony, which makes the confidence suffer a hard strike that the value comes all the way down to around 52.94. During the rest of the time, elements of scenes gradually stabilize and maintain their state to the end. Because the driving confidence of the ego is already at a low value, it recovers slowly compared to the previous, but at the same time, the amount of recovery is still gradually increasing. The whole process of the VC system functioning in scenario one can be seen in Figure 5.

The black vehicle in Figure 5 is the ego, while yellow blocks are representing the other traffic pedestrians. The color bar above shows the status of driving confidence. The brown line in the figure represents the normal environment state that the rainfall is 0 and riding on a standard road. Furthermore, the red line is the driving confidence of the completely same scenario when raining and the road type is an underpass. As can be seen, the two lines show a consistent change in trend, while the red line gradually has a lower value than the brown line as the situation continues to change. The cause for the red line’s value is the lower ${Re}_t$ determined by the high complexity of the ego surrounding. The result of scenario one indicates that VC’s basic functions can work properly. The verification score from 15 volunteers can be seen in Figure 6.

4.2. Scenario Two

The background of scenario two is the same as scenario one. Scenario two was built to verify the quantification of driving confidence of the VC model when vehicles change lanes frequently. So, the rest of the traffic participants are operating in a steady state to reduce the impact on the ego.

The ego vehicle takes the first lane change action at simulation time 10, and the lane change causes a current decrease in driving confidence. Furthermore, the vehicle stays in the current lane for about 200 simulation time. During this period, the confidence value successfully recovers to 100. This part of the result demonstrates that a single general lane change will not seriously impact driving confidence. Then the following behavior of the ego is set in violation of the usual regulations. It takes five continuous lane changes from 240 to 400 intervals 40 simulation time. Although the confidence value will start restoring at the end of each action, its recovery cannot catch up with the fast and frequent changing lane actions. The driving confidence suffers a major shock after the series of actions down to around 40. The simulation result of scenario two can be seen in Figure 7. The brown and red lines in the figure represent the same meaning as the result of scenario one. The brown line is for normal environmental conditions and the red is for rain and low visibility.

As can be seen in Figure 7, each lane change produces the same decrease in confidence according to Equation (15) under the same condition. Observe the result separately, significant differences can be found in the recovery rate for different confidence value states. Comparing the two results reveals the same phenomenon as the previous result that the complexity of the environment plays a role in determining changes in driving confidence to some extent. The verification score from 15 volunteers can be seen in Figure 8.

Both the results of Figure 6 and Figure 8 indicate to a certain extent that the VC model’s simulation of driver confidence values possesses a degree of credibility. Since it is impossible to obtain the driver’s actual confidence state, the model is only an abstract numerical reflection. Based on the feedback from the testers regarding the model’s performance, we believe it can be further optimized. Additionally, as the testers all have limited experience with autonomous driving, the model may largely reflect the confidence state of passengers who have just started using autonomous vehicles.

5. The Potential and Outlook

The VC can be conceived as a driver riding inside the autonomous vehicle. The early use of the VC can be integrated into the decision-making systems of autonomous vehicles, especially in systems with complex state machine structures. Decisions based on the state machine are generated by judging the current situation of the surroundings and the direction of the local goal. The driving confidence value can be introduced as an additional factor or additional condition for the state machine judgment. By verifying the current confidence level during the execution of typical actions, autonomous vehicles can access a more diverse range of execution options. For instance, without VC, when performing a lane change, the speed can be set within a wide range. By leveraging the VC, the vehicle can select a more optimal speed control strategy that considers the driver’s confidence level. This can involve maintaining the current speed when the confidence level is high (green) or reducing the speed to avoid further declines in confidence.

For the data-driven decision methods of the autonomous driving system, VC can still provide a certain degree of perfection support. The usual data-driven approach is using a machine or deep learning procedure to obtain the experience of good historical drivers’ behavior. The value of driving confidence can be used in both data collection and model learning stages. The confidence level can be recorded when human drivers operate the vehicle under specific situations and used as data for autonomous driving systems to learn how to respond to similar scenarios. Decision systems trained with driving confidence data can select behaviors that are more likely to maintain higher levels of driving confidence in a given scenario.

Advanced autonomous driving systems built with unsupervised learning approaches, such as reinforcement learning or deep reinforcement learning, can benefit from using VC as a tool to determine the system’s development direction. The effectiveness of the training method for an unsupervised model depends heavily on the setting of the reward function. The introduction of driving confidence values can be an important component of the reward function to constrain the behavior of vehicles. Although the model trained in this way may not necessarily be more efficient in executing the desired performance, its behavior will be more anthropomorphic.

In addition, the health bar and confidence display-related content have been shown in [25,26] to be of great help to the safety of human drivers working with autonomous driving systems inside future intelligent cockpits. AVs equipped with VC display in HMI can significantly increase the human’s confidence in autonomous driving while doing some personal jobs.

6. Conclusions

This paper performed a survey on drivers’ confidence while riding in an AV. By analyzing the results, we summarize the main associations between AV and driver confidence. A framework for underlying confidence changes and the quantitative models related are constructed. The VC model realizes the modeling of other traffic participants’ interference, the ego vehicle’s unexpected status, and the environmental complexity. By analyzing the relationship between the various influencing factors, it achieves a closed-loop feedback quantification of the real-time confidence value. The VC model successfully solves the problem that there is currently no solution to quantitatively assess the real-time behavior state of the autonomous driving system in the field of auto-driving anthropomorphic. To demonstrate the effectiveness of this revolutionary structure, two usual traffic scenarios were created to test the confidence values output by the VC. The analysis of the result verified that the VC model has a good function in quantifying the vehicle’s confidence, and the experiment results also confirmed the extensibility and great application prospects of the model. In future experiments, the VC model and the system incorporating its functions can be subjected to more engineering-oriented tests. Researchers can mount the model on a real autonomous vehicle and configure a safety officer to experience the performance of the vehicle in handling special scenarios. By comparing the vehicle status under the low VC output and the safety officer’s feeling, the effectiveness of the model’s validation on the real vehicle can be evaluated.

We believe that continued research on VC can further enhance the safety of AVs while improving their intelligence, and can maximize its potential in a human-experience manner. The designed structure of the VC can also provide technical references for quantification in other related fields, such as constructing the users’ confidence model in the intelligent cockpit, smart battery management systems or energy control systems.