^{1}

^{*}

^{1}

^{1}

^{2}

^{2}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

In this paper, we describe a new framework to combine experts' judgments for the prevention of driving risks in a cabin truck. In addition, the methodology shows how to choose among the experts the one whose predictions fit best the environmental conditions. The methodology is applied over data sets obtained from a high immersive cabin truck simulator in natural driving conditions. A nonparametric model, based in Nearest Neighbors combined with Restricted Least Squared methods is developed. Three experts were asked to evaluate the driving risk using a Visual Analog Scale (VAS), in order to measure the driving risk in a truck simulator where the vehicle dynamics factors were stored. Numerical results show that the methodology is suitable for embedding in real time systems.

Driving in an urban environment is always a risky and complex experience [

The United States Department of Transportation [

Consequently, one key issue is how to estimate the driving risk in such an environment [

Modern truck vehicles are completely sensorized and could incorporate as many as 70 electronic control units (ECU) for various subsystems [

Therefore, to complete all the elements, we have to consider the definition of driving risk [

The data used in this paper come from the “Intelligent cabin truck for road transport” (CABINTEC) project funded by the Spanish Ministry of Science and Innovation, and the European Union. This project is focused on driving risk reduction for professional drivers (mainly focused on trucks and buses). The three main aspects of traffic safety are considered: road, vehicle and driver. Road and vehicle data are obtained from a highly realistic simulator and the hands of the driver are surveyed to measure their position.

This paper is organized as follows: Section 2 introduces the general framework of the problem at hand. In Section 3, we present the computational framework for the fusion and selection of experts. Section 4 describes the high immersive experimental set-up. In Section 5 the numerical experimentation is carried out and results are shown. Finally, Section 6 presents our conclusions.

Driving is a dynamic activity that involves changes in the values of several variables (speed, brakes, lights, and so on) along time. The values of these variables in a precise instant will determine the actions made by the driver. Formally, let us define an action as the values of a set of variables _{1}, …, _{k}

Suppose that we have a set of actions (a population) made by a driver. This population includes past actions and it could also include future or potential actions. We assume that the size of the population of actions,

Throughout this work, we will use indistinctly the term “variable”, used mainly in the ITS terminology, and the term “characteristic”, more common in the quality management field. Both terms are equivalent from a statistical point of view.

In Intelligent Transportation Systems (ITS) we can follow a similar approach by assigning to each action a risk level determined by a traffic safety expert. Let us call _{i}_{TH}_{i}_{i}_{TH}_{i}_{i}_{i}_{i}

In the literature, it is common to assume that customer's evaluation will be a function of several characteristics _{1},…,_{k}_{i}_{1},…,_{ik}_{TH}

Notice, that in this approach, the function _{j}

These weights can be considered as measures of the relative importance of characteristic _{i}

In order to adapt and extend this quality model to measuring driving risk, from now on, we will refer to customers as traffic safety experts. In this way our effort will be dedicated to present and develop a methodology to calculate the weights assigned to each variable describing an action taking into account the perception of the action risk provided by an expert, considering that different actions may have different weights for the characteristics. The methodology has been developed taking into account that the final implementation will be embedded in a system that should take the data and obtain results in real time conditions.

For the sake of simplicity in the following description of the model we will consider a single expert assigning the risk to each action. Let us call _{i}_{1},…,_{ik}_{TH}_{i}_{TH}_{i}_{i}_{ij}

Regression models following a similar reasoning have been presented in [_{1},…,_{k}

This methodology also overcomes the traditional consideration of the expert's evaluation of an action. Within the classical settings the whole set of correlations among the characteristics and the risk values provided by the expert are explicitly calculated and used to detect the relevant variables. Under our approach these correlations are implicitly taken into account when calculating the weights for each action. In this way more information is achieved as in addition to the raw correlations, we obtain the explicit important of each characteristic within each action.

A motivation to use a fusion of judgments is the minimization of errors due to the experts. It is expected that the inclusion of different criteria (experts) will minimize erroneous appreciations coming from some of the experts. As mentioned above, due to possible inconsistencies among different experts, it is interesting to merge all the information or opinions available into a single one [_{j} is the weight assigned to expert _{j}

The weights _{j} should be high for the most confident experts and small for inconsistent experts. In addition:

In fact, a possibility is to assign the _{j} values from our knowledge of each expert confidence. If no

In this section we describe how to compute both, the weights _{ij}_{i}

For the sake of completeness, we include next a description of the algorithm used to calculate the weights described in the previous section. From a mathematical point of view, our initial hypothesis is that there is a function _{i}_{i}_{i}_{i}_{1},…,_{ik}

Thus, our model can be locally linearly approximated as
_{1}, ,_{N}

Given an action _{i}_{i}_{1},…,_{ik}_{i}_{i}_{i}_{i}

Given an action _{i}_{(1)},…,_{(}_{l}_{)}, where (1),…,(^{i}^{i}

Notice that we may build _{i}_{i}

Regarding the choice of the number of neighbors

Our aim is to estimate each component of the matrix

We estimate the weights that each action assigns to each characteristic with the information obtained from similar actions. We choose the set of “similarities” based on the nearest neighborhood estimate. This methodology is applied for each expert involved in the evaluation. Once the methodology has been applied, we can obtain an estimation of the expert judgment, _{i}^{i}ŵ_{i}

The proposed methodology presents several advantages:

When the decision maker needs a single common index, a scalar measure to summarize the performance, we can define:

We have estimated each component _{ij}

We define a vector of weights for each action; therefore we are implicitly defining the importance given by the expert to each characteristic of the action. Notice that working with these weights as data we may define new relations among the data.

The use of neighbors helps to minimize the simulator errors. In this regard, given that neighbors are chosen using a similarity measure, it is expected that actions including variables with a high acquisition error will be discarded and only actions similar to the current driving situation will be selected. If the acquisition error appears in one variable within the current action, given that the similarity measure takes into account the whole set of variables involved, the use of multivariate actions will minimize its influence.

Let us consider a simple explicative example of how the described methodology calculates the nearest neighbors and the estimation of the judgment for a new action that has not been previously judged. For the sake of simplicity, let us consider that the number of variables describing an action is two, namely, speed and steering wheel angle (SWA).

First of all, we have to decide the number of nearest neighbors that are to be calculated. As already mentioned we will use the truncated integer value of
_{3} and _{5}) we can estimate the weights corresponding to the unjudged action, _{UA}_{UA}_{1}, _{UA}_{2}] (Step 3 in

These weights are then used to predict the judgment of the unjudged action _{UA}_{UA}_{1} + 25_{UA}_{2} =40.7. The same procedure can be used to estimate the predicted risks _{i}_{i}

We can define, the mean quadratic error (MQE) for an expert as:

Notice that the more consistent the judgments of an expert are, the less value for MQE is. From this definition of MQE, we can build the weights _{i}_{i}

It is important to remark that with these definitions the following properties hold:

The most confident experts, that is, those with the smallest MQE, will have the highest _{i}

The sum of the _{i}

These two properties are exactly the ones specified in Section 2.2.

Using these weights, the fusion of experts is straightforward following the methodology described in Section 2.2. Regarding the selection of experts, the _{i}

The methodology was applied to measure the weights of the parameters involved in the driving risk evaluation. As already mentioned, the data used in this paper come from the CABINTEC project. Within this project, a truck cabin simulator has been built. The simulator is a real truck cabin placed on pneumatic actuators and computer controlled, as shown in

Driving dynamics and kinematics are simulated in order to obtain realistic driving data. To this aim, 12 professional drivers were employed in the project. No previous information was given to the driver, so natural driver behavior was expected. Each driver performs sessions during a minimum of ten minutes. Information on 27 variables was collected at the simulator, among which we can mention the speed of the truck, the revolutions per minute (RPM) of the engine, the angle of the steering wheel (SWA), position of the truck on the road, the slope of the road, the position of the pedals, etc. Each variable provides a data point every 5 milliseconds, so for each session more than 50,000 data points were collected. Also, to provide more information to the experts, two videos per session were collected, the first one with the images projected to the driver and the second in the truck cabin showing the driver's behavior.

Three independent and extern safety experts of the Spanish Royal Automobile Club (“RACE”) evaluated the risk level at each point assigning a value in a Visual Analog Scale (VAS) according to their individual perception. A VAS is a response scale used to measure subjective characteristics or attitudes that cannot be directly measured. When using VAS, experts specify their subjective level of risk by indicating a position along a continuous line whose length goes from zero to 100 [

The background of each expert is the following: Expert 1 has high experience in this kind of evaluations and a strong expertise in traffic safety. Experts 2 and 3 are novel to this kind of evaluation, although they also have experience in traffic safety. The scale used adopts values from zero to 100. This method was chosen because it is a simple method that correlates well with other descriptive scales, it has good sensibility and liability, and it can be easily repeated.

The experts evaluate the driver's behavior during 10 minutes, which corresponds to 35,000 clock cycles [

It is important to remark that the number of actions amounts to almost 60 per second. This means that we have many similar actions and, if an acquisition error occurs in one action, its influence in the overall performance of the methodology will be very low. Notice also that expert number one (the blue line) presents an evaluation of the risk more stable (with lowest variability) than the evaluation of the risk built by the other two experts (the green and pink lines).

In this section we present the results of applying our methodology to the CABINTEC project datasets. We have divided the data corresponding to each expert into two disjoint subsets:

Training subset: It is used to estimate the weights. The size of this subset is denoted by

Testing subset: It is used to validate the model established. The size of this subset is denoted by

As described in Section 3, the number of neighbors has been chosen as the maximum value between the value

In

It is important to remark that Expert 1 consistently obtains the highest weights. This is coherent with the _{i}

In real situations, our training set will be smaller than the testing set, which corresponds to the infinite possible new actions that may occur in the future. As already mentioned, the methodology has been designed to be embedded within a system that acquires data and make decisions in a real time environment. As a consequence, the most realistic situation is the one that considers a small training set compared to the testing set. This is the case for n = 10,000 and Tn = 20,000. In this situation the _{i}

The methodology described in Section 2 can be used to determine the relevant characteristics for a given expert or fusion of experts. We mean by relevant those characteristics whose _{ij}_{ij}_{i}

Notice that the _{ij}

As already mentioned, Expert 1 has the _{i}_{i}

In

Next we have measured the maximum absolute error and the mean square error for different sizes of the training set.

In

Again for Experts 2 and 3 we have measured the maximum absolute error and the mean square error for different sizes of the training set. Numerical results are shown in

The methodology captures accurately the scoring pattern followed by Expert 1, while Experts 2 and 3 show larger variability in their perceptions of driving risk. Notice that this is confirmed in

For Expert 1, the characteristics with the highest values for the median and the third quartile (Q3) are Gear, Steering Wheel angle and Speed. Regarding speed, it is a choice of this expert to assign a high level of risk to this characteristic. This level does not depend on the legal speed limit, but on the adequate speed level according to the driving situation and the expert's judge. Notice that Expert 1 has the most similar results to the ones obtained from the fusion of experts. This is natural since the _{i}

Regarding Expert 2, the characteristics with the highest values for the median and the third quartile (Q3) are Gear, Lateral position and Speed. Again, the characteristics Gear and Speed appear as some of the important variables. This expert gives also importance to the Lateral position of the truck. The Lateral position is measured with respect to the right lane border.

For Expert 3, the characteristics with the highest values for the median and the third quartile (Q3) are Speed, Lateral position and Accelerator proportion. The Accelerator proportion represents the percentage of acceleration at the given instant (0 means no acceleration and 100 means full gas). As a conclusion, all the three experts agree in the fact that Speed is a relevant characteristic regarding risk.

The results described in this section follow the current research trends in the automobile industry. The development of safety traffic systems based on real time analysis of variables has been implemented by companies such as Volvo and Mercedes. In this regard Volvo has recently introduced a new technology feature, the so called City Safety. The system has a closing velocity sensor that helps to prevent low speed collisions. [

Concerning Mercedes, this company has developed the Attention Assist System, which based on the analysis of 70 variables, tends to detect the driver's driving style in order to measure his level of drowsiness [

The system developed in this work goes a step further, as it can be used to prevent general risks and is not focused in particular cases (such as drowsiness or low speed collisions). Another novelty is that the system described has been developed using trucks as the basis of the simulation. Traditional safety systems are usually implemented in executive sedan models but it should be taken into account, as already mentioned in the introduction of this paper, the consequences of fatal trucks accidents in urban areas are usually more dangerous than accidents involving sedan models.

This paper presents a methodology to combine and select experts' judgments for the prevention of driving risks in a cabin truck. The proposed technique allows the calculation of the influence of each characteristic over the final driving risk value, for each expert. This is an important contribution in order to analyze the causes of risky situations during the driving task. In addition, the methodology described in the paper allows the calculation of the explicit weights assigned to characteristics involved in driving risk. Data have been obtained from a truck simulator where the values of the characteristics involved in the vehicle dynamics were stored. Three experts were asked to evaluate the driving risk using a Visual Analog Scale (VAS). We have used the evaluation of each expert to find the weights that he implicitly assigns to every dimension (characteristic) of the given situation (action).

It is important to remark that the methodology emulates the experts' judgments taking into account that the final implementation will be embedded in a system that should take the data and obtain results in real time conditions without the presence of the real experts. Therefore, the methodology has the capacity of being used in a real time scenario.

Particularly, the methodology presented in this work allows the ordering of the experts' judgments according to their relative confidence. Moreover, the characteristics involved in driving risk can also be sorted in order to select the most relevant ones. Notice that

This work has been partially funded by the following projects: CABINTEC project, funded by Spanish Ministry of Science and Innovation and FEDER funds (European Union), PSE-370000-2008-2. 2007-2011; Vulcano (ref 442808215-8215-4-9) funded by Spanish Ministry of Science and Innovation; RIESGOS-CM project, Ref. CAM s2009/esp-1594, funded by the Government of Madrid; EDUCALAB project, Ref. IPT-2011-1071-430000 and AGORANET project, Ref. IPT-2010-23-430000, funded both by the Spanish Ministry of Science and Innovation; and project F122 from Universidad Rey Juan Carlos.

Algorithm to estimate _{ij}

Truck simulator used in the experiments.

Immersive environment for the simulator.

Quantitative evaluation of the risk by the experts.

Risk level for the fusion of experts (n = 10,000).

Driving risk evaluation given by Expert 1 (n = 25,000 and Tn = 5,000).

Driving risk evaluation given by Expert 2 (n = 25.000 and Tn = 5.000).

Driving risk evaluation given by Expert 3 (n = 25,000 and Tn = 5,000).

Five judged actions and a new action whose judgment is not available (NA) in advance and has been predicted.

_{i} | |||
---|---|---|---|

Action 1 | 80 | 0 | 21 |

Action 2 | 50 | 90 | 60 |

Action 3 | 60 | 30 | 40 |

Action 4 | 80 | 30 | 54 |

Action 5 | 70 | 20 | 39 |

Unjudged Action | 68 | 25 | NA |

Estimated weights for each expert using different training sizes,

Expert 1 | 0.35 | 0.37 | 0.67 | 0.80 |

Expert 2 | 0.33 | 0.29 | 0.17 | 0.10 |

Expert 3 | 0.32 | 0.34 | 0.16 | 0.10 |

Relevant characteristics for the Fusion of Experts (

Gear | 0.11 | 0.12 | 0.19 | 0.27 | 0.45 |

Steering wheel angle | 0.09 | 0.08 | 0.12 | 0.18 | 0.48 |

Lateral position | 0.09 | 0.00 | 0.08 | 0.17 | 0.26 |

Accelerator prop. | 0.12 | 0.00 | 0.10 | 0.22 | 0.44 |

Speed | 0.12 | 0.03 | 0.14 | 0.23 | 0.45 |

Brake prop. | 0.06 | 0.00 | 0.01 | 0.06 | 0.47 |

Weights for the experts (n = 10,000).

_{i} | |
---|---|

Expert 1 | 0.80 |

Expert 2 | 0.10 |

Expert 3 | 0.10 |

Results for Expert 1 with n = 25,000; n = 20,000; n = 15,000 and n = 10,000.

max (abs. error) | 1.05 | 1.28 | 1.42 | 1.46 |

standard dev. | 0.35 | 0.44 | 0.47 | 0.51 |

mean sq. error | 0.36 | 0.55 | 0.68 | 0.73 |

Results for Expert 2 with n = 25,000; n = 20,000; n = 15,000 and n = 10,000.

max (absolute error) | 1.26 | 2.14 | 4.11 | 5.63 |

standard dev. | 0.42 | 0.72 | 1.40 | 1.93 |

mean squared error | 0.36 | 0.71 | 2.72 | 5.85 |

Results for Expert 3 with n = 25,000; n = 20,000; n = 15,000 and n = 10,000.

max (absolute error) | 1.64 | 2.43 | 3.62 | 5.14 |

standard dev. | 0.56 | 0.82 | 1.34 | 1.68 |

mean squared error | 0.38 | 0.59 | 2.91 | 5.91 |

Relevant characteristics for Expert 1.

Gear | 0.12 | 0.10 | 0.19 | 0.30 | 0.49 |

Steering wheel angle | 0.10 | 0.09 | 0.14 | 0.21 | 0.56 |

Lateral position | 0.08 | 0.00 | 0.07 | 0.15 | 0.11 |

Accelerator prop. | 0.13 | 0.00 | 0.11 | 0.25 | 0.51 |

Speed | 0.11 | 0.03 | 0.14 | 0.24 | 0.43 |

Brake prop. | 0.05 | 0.00 | 0.01 | 0.05 | 0.40 |

Relevant characteristics for Expert 2.

Gear | 0.08 | 0.06 | 0.13 | 0.19 | 0.36 |

Steering wheel angle | 0.02 | 0.01 | 0.02 | 0.04 | 0.17 |

Lateral position | 0.14 | 0.00 | 0.12 | 0.26 | 0.54 |

Accelerator prop. | 0.02 | 0.00 | 0.01 | 0.04 | 0.13 |

Speed | 0.10 | 0.02 | 0.12 | 0.20 | 0.46 |

Brake prop. | 0.08 | 0.00 | 0.01 | 0.08 | 0.64 |

Relevant characteristics for Expert 3.

Gear | 0.02 | 0.01 | 0.03 | 0.05 | 0.11 |

Steering wheel angle | 0.04 | 0.04 | 0.06 | 0.09 | 0.30 |

Lateral position | 0.09 | 0.00 | 0.08 | 0.17 | 0.36 |

Accelerator prop. | 0.08 | 0.00 | 0.06 | 0.14 | 0.37 |

Speed | 0.10 | 0.02 | 0.12 | 0.20 | 0.45 |

Brake prop. | 0.05 | 0.00 | 0.01 | 0.04 | 0.40 |