1. Introduction
Customer-driven vehicle attributes include properties, such as a vehicle’s steering feel, handling, and ride qualities. Currently, testing such properties is primarily performed on-road by car manufacturers, with test drivers from the evaluation team driving the vehicle prototypes at various stages of development, subjectively judging their behaviour, and comparing across alternative designs. Typically, these design alternatives involve changes to costly prototypes that can require many hours of work. There are also associated risks of inaccurate evaluations since drivers are not comparing alternative designs in close temporal proximity.
The virtual prototyping of vehicles aims to lower the burden and expense by replicating the conventional evaluation methods in a virtual environment, facilitated by simulators. Testing the vehicle properties in the simulator does not require physical prototypes, and arbitrary changes to vehicle designs can be applied without time delays. Compared to real driving, simulators offer a safer, repeatable, more versatile and less costly environment to measure driver perception and behaviour. However, there are physical limitations in the visual and vestibular simulator cueing subsystems, which degrade the fidelity and restrict its application [
1,
2].
There is a relatively sizeable literature on real-world subjective assessments of handling and steering feel of vehicles. The key factors have been found to be subjective measurements over the steering response: window, modulation, torque, effort and handling stability and controllability characteristics [
3]. Furthermore, there have been efforts to correlate the subjective assessments to the objective measurements acquired from computer-aided engineering, steering robots or logged data, to help in development and testing of new systems [
4,
5,
6].
In spite of the available real-world studies, the ride, handling and steering qualities are sparsely studied in driving simulators. It is feasible to study some of the vehicle characteristics (e.g. vehicle field of view) in static driving simulators without a motion platform; however, the vehicle characteristics that are perceivable through stimulation of the human vestibular system always require dynamic simulators so that the motion cues are available.
In dynamic simulator-based related research, different steering system friction, damping, inertia and stiffness characteristics have been studied, and correlations between the subjective ratings and objective measurements have been observed [
4]. Drivers’ preferences of steering wheel torques between the real vehicle and simulator have been studied [
7]. They have reported that the drivers preferred stiffer steering wheel response for increasing speeds in both environments. The pattern of increasing preferred stiffness disappeared above driving speeds of 40 km/h when the simulator motion platform was disabled, suggesting an interaction between perceived lateral accelerations and steering feel preference.
Baumgartner et al. studied the ability of drivers to distinguish between the acceleration profiles resulting from different powertrain setups [
8]. The just noticeable difference (JND) method was used on different down-scaled reference acceleration profiles, and the drivers’ perception in the context of drivability (longitudinal acceleration of the vehicle) was evaluated. They reported that the JND increased at higher acceleration intensities. In the literature, it has been consistently reported that the duration and shape of acceleration profiles and whether there are single or multiple sensory stimulations affect the absolute perception threshold [
9,
10].
Driver perception regarding vehicle response in various tyre-road interactions has been studied by adjustments to the tyre models in a simulator [
11]. Four tyres with different characteristics were compared to a reference tyre in a qualitative assessment by expert drivers. The results showed that the different characteristics were perceived with different intensities, and some were described by drivers in a contradictive manner. It was concluded that some of the tyre characteristics (e.g. thread stiffness) could be tested in dynamic driving simulators.
A study of the dynamic performance of heavy vehicles has been conducted [
12]. They investigated the correlation between the subjective evaluations of the vehicles with objective measures and compared the vehicle performance and stability. From the analysis of 50 drivers, the differences between the performance of the various vehicle configurations were perceived. They also found a strong correlation between the perceived and objective performance of the vehicle. They concluded that simulator usage is a complementary tool for the existing test chain rather than a replacement.
Among the few existing papers on simulator-based testing of vehicle design, a standardised evaluation of the ride, handling and steering feel qualities is missing. Besides, no research has looked at the effect that simulator motion cueing subsystems might have on the evaluation of these vehicle attributes. To this end, the manipulations to the ride height (RH) of the air suspension of a Range Rover Velar prototype vehicle were selected as the first independent variable. It was identified as a manipulation that causes change to the driven attributes in a subjectively comparable nature and magnitude to those involved in a typical on-road driven attributes testing. It would not be discernible to the drivers before starting to drive and was readily available in existing vehicle dynamics models from Jaguar Land Rover (JLR), which could be implemented in the University of Leeds Driving Simulator (UoLDS) simulator, where the experiment was performed.
The second aim of this study was to consider the effect that motion characteristics of a dynamic driving simulator might have on the assessments of vehicle attributes, by means of subjective and objective behaviour measurements. Therefore, the manipulation of the simulator motion platform (MP) workspace size was selected as another independent variable. The MP workspace defines the motion envelope to represent the vehicle motions in the simulator, which directly affects drivers’ perception of motion.
The following sections explain how the experiment was designed, alongside the introduction of the driving manoeuvres, test vehicle and measurement methods. This is followed by the analysis of the drivers’ subjective ratings and objective performance results, a discussion and the conclusion.
2. Materials and Methods
2.1. Experimental Design
The ride height (RH) of air suspension had three levels (low, standard and high). The individual ride heights were selected during real-world piloting and set as:
Low: −8 mm compared to a standard ride height
Standard: the standard vehicle ride height
High: +8 mm compared to a standard ride height
The real-world piloting confirmed that the ±8 mm ride height manipulation had impacts on the driven attributes in a subjectively comparable nature and magnitude to those involved in typical on-road driven attributes testing. Piloting in the driving simulator confirmed that the ride height manipulation had a similar impact on the simulator.
Three motion platform configurations (small, medium and large) varied the motion envelope in the surge and sway degrees of freedom (DoF) provided by the sliding rail, and the maximum yaw angle of the hexapod.
To increase the power of experimental manipulations, a repeated measure experiment design was used, where the same participants took part in the nine (3 × 3) combinations of the independent variables; subjective ratings and drivers’ objective performance were collected.
The experiment was designed in a nested way with the comparisons between RHs as the primary focus for drivers and the MPs as the secondary focus. This was because the pilot testing showed that the differences between the RHs were more indistinguishable compared to the MP configurations. This was because the vehicle motions across RHs were generated in the simulator without any extra attenuation between them, while the vehicle motions in MPs were attenuated differently for each MP.
2.2. Drivers, Manoeuvre and Vehicle
Six drivers, all JLR employees with experience in assessment of vehicle driven attributes, took part in the experiment. The low number of participants was due to JLR personnel availability constraints as not many JLR drivers do these tests, and untrained drivers could not be employed, since the drivers’ perception and performance might differ.
The road used for the experiment, shown in
Figure 1, is a 60 mph narrow rural road, including both straight segments and curves of varying radii. It is part of a public road circuit, starting and ending at the JLR Gaydon site, Coventry, UK. For the simulator, a LIDAR 3D scan of the road circuit was converted for use with OpenSceneGraph.
A Dassault Systèmes SimPack multi-body simulation model of the dynamics of the Range Rover Velar vehicle developed by JLR was used in the study, see
Figure 2. This model executed on 8 cores of a dual Xeon CPU 16-core Concurrent iHawk (Duluth, GA, USA) hardware platform.
The yaw rate response gain of the vehicle model in the 80–100 km/h speed range, for the three ride heights, was validated against real-world data.
The ride heights were modelled by changing the corresponding parameter in the SimPack model. The model incorporated the Delft tyre model, and the road surface was represented as a regular curve grid (CRG) at 30 cm resolution, with a constant 0.95 friction.
2.3. Measurement Metrics
2.3.1. Subjective Measures
A questionnaire was developed to assess the driven attributes of the Range Rover Velar vehicle to describe the properties of a vehicle as given in
Table 1. The centre point of the rating line was “appropriate for this vehicle”, which allowed a comparison between the real and virtual driving experience.
The primary ride assessed the low-frequency vertical movements of the vehicle caused by large and obvious, continuous road undulations, which results in a moderate range of suspension travel. The secondary ride assessed the quality of the vertical vibrations caused by the road texture and road disturbances. Typically, in low RH, drivers get more road feel, a constrained stiffer feeling, while the high RH results in a more comfortable, floaty and settled feeling [
13,
14].
The steering properties consisted of questions about steering window, gain linearity and effort. Typically, for a low RH, higher control effort is required from the drivers, while a high RH results in more relaxed behaviour [
13,
15].
The last set of questions revolved around standard handling properties. Ride quality and handling have always been a compromise [
14]. Typically, in low RH, drivers get better handling and improved traction, while the opposite is true for the high RH [
13].
Lower RH brings the centre of gravity of the car lower (closer to the ground), reducing weight transfer during acceleration and braking, which makes the car more responsive by decreasing the body roll and pitch.
2.3.2. Objective Measures
The objective measurement methods are based on the “Utility Triplet” [
16]. The utility triplet has been proven to be a useful, structured approach for this type of behavioural fidelity assessment and includes three major categories: aggregate performance, time series and driver model [
17].
The aggregate performance metrics reflect the performance of a driver in terms of general qualities of driving, and it is related to the driving task definition, consisting of lap time, speed variability and maximum lateral position deviation. In this study, drivers were instructed to keep a constant speed, and lower lap time in a successful attempt showed improved performance since less deceleration was needed. A lower speed variation also showed better performance since drivers were asked to keep a constant speed. The lateral position deviation was calculated based on a reference trajectory, which was the average trajectory of all drivers across all conditions.
The time-series metrics included the number of steering reversals, a measure of drivers control activity [
18], where a reduced steering reversal rate showed a more relaxed style of driving. The steering reversal rate amplitude was selected to be 1 °/s and 10 °/s to capture small and large steering adjustments [
19]. The other time-series metrics are related to the maximum vehicle motions in different longitudinal, lateral and yaw directions. The higher speed of the vehicle usually increases the vehicle body roll [
20].
The desired path yaw rate error (DPYRE) model was used to assess the driver’s performance [
21,
22]. It uses three parameters (delay time, preview time and gain) presented in Equation (1), and the reference trajectory to predict the driver behaviour (steering rate).
where
is the rate of change of the steering wheel angle,
is a gain constant, the yaw rate error
is the deviation between current yaw rate
and desired yaw rate
, and
is a response delay. In Equation (1),
is obtained by looking ahead a preview time
on the desired path to determine the desired vehicle position at that point in time, and then determine the desired yaw rate
as the yaw rate that would take the vehicle to that desired vehicle position in the time
. A brute force grid search was used to fit all nonlinear parameters (e.g.,
), and for each tested combination of these nonlinear parameters, all linear parameters (e.g.,
K) were estimated using least-squares fitting.
2.4. Experimental Procedure
The drivers attended the UoLDS and drove for three sessions, each with a different motion platform size and made a comparison between the platform size-ride height pairs:
Small-low, small-standard, small-high
Medium-low, medium-standard, medium-high
Large-low, large-standard, large-high
The above sessions were presented to two of drivers in small-medium-large MP group order, to two drivers in large-medium-small order and to the final two drivers in medium-small-large order, and drivers were randomly assigned to each group. The order of the RHs within each of the above MPs was counterbalanced per driver.
The questionnaire asked participants to rate the three ride height configurations relative to each other by placing a cross on a continuous line after driving each configuration. Next, they had a break and started with the next motion platform session. They did not know which RH or MP they were experiencing. They were familiar with real-world testing of the vehicle in a similar high-friction condition and driving task.
2.5. Apparatus
2.5.1. Motion Platform
The UoLDS motion platform built by Bosch Rexroth, Lohr a. Main, Germany consists of a 6 DoF hexapod platform mounted on a 2 DoF sliding rail. A maximum of 15 degrees of yaw and 0.1 m of the surge, sway and heave were selected, which the hexapod can provide simultaneously in all DoFs. The hexapod also replicated the vehicle roll and pitch motions, which were always less than 5 degrees, in a one to one manner. The MP envelope was varied in surge, sway and yaw workspace. The motion platform configurations are shown in
Table 2. Full motion system characteristics are available in [
23].
2.5.2. Motion Cueing Algorithm
The driven attributes evaluation relies on the subjective evaluation of vehicle motion, and therefore the motion cueing algorithm must be carefully chosen. The model predictive control (MPC) algorithm has been shown to have good performance and to be preferred by drivers [
24,
25,
26], and therefore a version of this algorithm was used for this study [
23]. The longitudinal and lateral accelerations in the simulator were generated through transient linear motions of the hexapod and sliding rails and the sustained acceleration of tilt coordination. The three motion platform accelerations can be summed (Equation (2)) to calculate the total acceleration the driver senses in the simulator.
The plant model for each channel has three control inputs
and nine outputs corresponding to the hexapod and rail translations, the hexapod tilting and their summation
. The complete plant model for the surge/pitch channel is presented in Equation (3).
where
and
are the row vectors extracted from the first row of
and
The state vector (manipulated variables) of the plant consists of the states of the otolith model including , for each of the rail, hexapod and tilt; the position and velocity states of the rail and hexapod and the tilt pitch angle . The output vector (tracking variables) consists of perceived acceleration, velocity and position represented by the rail ,, and hexapod ,, ; perceived acceleration represented through tilting and tilting degree ; total perceived acceleration .
Nonlinear scaling of the vehicle input motions was used in this experiment based on the method in [
27]. The scaling for the lateral motion of the vehicle is shown in
Figure 3. The same scaling function was used for the longitudinal vehicle motion, although the maximum input acceleration was set to 9
, because of higher accelerations during accelerating and braking.
For each MP workspace size, the MCA was tuned in the time domain using offline trial and error, to avoid false cues, while exploiting the MP workspace. The main measure for the comparisons was the root mean square (RMS) of the difference between the vehicle motion (vehicle dynamic output—before scaling) and MCA output acceleration (including scaling) and the total excursion of the motion base. The motion response of the final configurations is presented in
Table 3, with the increase of MP size resulting in a reduction of RMS error.
Considering the range of tilt-coordination parameters used in the literature [
28,
29,
30,
31], tilt setting candidates were reviewed during the experimental design, and angular velocity and accelerations of 4
and 4
were found to be acceptable based on the normal and professional drivers’ comments who participated in the design and piloting steps.
3. Results and Discussion
3.1. Subjective Evaluations
Due to the high number of grouped questions, a separate reliability analysis was carried out for every subscale of the ride, steering and handling questionnaire. Cronbach’s α [
32] allowed the grouping of some of the questions, as shown in
Table 4.
A repeated measure analysis of variance (ANOVA) established the main and interaction effects of RHs and MPs on the subjective ratings, which are shown in
Table 4. Significant effects
were presented, as well as trends
due to the limited sample of drivers. The small number of professional drivers was representative of the usual vehicle design evaluations; however, it limited the statistical power of the analysis; hence, the effect sizes are also reviewed in the next sections. To verify the main effects, detailed ANOVA contrasts and post hoc Bonferroni pairwise comparison tests were performed. The subjective rating results for the questions are presented in
Figure 4, in which a 0.5 value corresponds to the “appropriate for this vehicle” subjective rating.
There was a significant effect of MP on the vehicle’s ride attribute quality
,
. Contrasts revealed that ratings of medium and large MPs were significantly different from the small. From the
Figure 4 ride plot, the low and standard RH were rated quite similar by the drivers, where they found them constrained and stiffer, and they found high RH settled and floaty and more appropriate for the vehicle, as expected. Moreover, the medium MP was rated as the most appropriate for this vehicle in all RHs.
There was a trend of RH on the vehicle’s steering gain away from centre attribute quality, F(2,10) = 3.34, p = 0.07, with the ratings of the low and high configurations showing indications of being different from the standard. The low and high RHs were more nonlinear to drivers, and the standard RH was more linear and appropriate for the vehicle. Moreover, looking at plots, an inverse relation was observable between the increase of MP size and RH.
There was a trend of RH on the vehicle’s handling roll attribute quality, F(2,10) = 3.09, p = 0.08, where the ratings of the high configuration showed signs of being different from the low. From the plot, it was observable that the drivers found that the transient roll acceleration/deceleration was less for low and more for high RHs, which showed they had distinguished them correctly. Moreover, an increase in the value of the vehicle roll was observable with the increase of MP size for all three RHs.
There was a significant effect of RH on the vehicle under/over steering attribute quality, F(2,10) = 4.27, p = 0.04, where the ratings of the low configuration were significantly different from the standard. From the plot, the low and high RHs were more oversteering to drivers, and the standard RH was more understeer and appropriate for the vehicle. Moreover, an inverse relation was observable between the increase of MP size and RH. There was a significant interaction effect between the type of MP and the type of RH used, , . It was significant when comparing low to standard RH for medium compared to large MP.
For the steering window and effort grouped questions, drivers seemed to have chosen a larger MP with increasing RH as more appropriate. To find out how the different RHs and MPs affect the drivers’ subjective ratings of driven attribute qualities, a summary of findings is represented in
Table 5. It was observable that for most steering and handling attributes, in the low RH, the small MP was rated as the most appropriate for this vehicle, and in the high RH, it was the large MP.
3.2. Objective Evaluations
The whole track was split into sections, and each section was analysed separately. The road and sections are shown in
Figure 5. Due to the similarity in nature of many of the road sections and to reduce the number of reported results, the sections were grouped into straights and curves. The objective metrics of a group section was calculated from the average value of all sections belonging to that group, as per
Table 6.
3.2.1. Aggregated Performance
No significant effect of MP or RH on speed variation was observed. From the effect sizes, the speed variation slightly decreased with increasing MP size for standard and high RHs. However, the effect sizes were small to medium and, therefore, should be treated with caution. The summary of results is represented in
Table 7.
The lap time in the straight and curved sections showed: During the straights and curves, there was a significant effect of MP , on lap time, wherein both sections, the lap time in the large configuration was significantly higher than medium, and it increased with the increase of MP size. During the straights, there was an interaction trend between the type of MP and the type of RH on the lap time, , , where it was significant comparing low to standard for small compared to medium, and significant comparing low to standard for medium compared to large.
The lateral position deviation in the curve sections was significantly higher for the small MP as compared to the medium and large,
,
, which decreased with the increase of MP size. Moreover, the lateral position deviation in the straights was significantly higher for the low RH as compared to the standard and high,
,
, reflecting the deteriorated performance of drivers while experiencing the low RH. During curves, the lateral position deviation was significantly higher for the small MP as compared to the medium and large,
,
. The summary of results is represented in
Table 8.
3.2.2. Time Series
In
Table 9, the repeated measure ANOVA results of time-series metrics in both curves and straights are shown for both RH, MP factors and their interactions.
Curve Road Sections
The DPYRE preview time was significantly higher for the small MP as compared to the medium and large, , . There was a trend effect of RH on the longitudinal velocity F(2,10) = 2.82, p = 0.08, where there was a trend for the low configuration to be driven faster than the high. There was also a significant interaction effect between the type of MP and the type of RH used on the longitudinal acceleration, , , when comparing standard to high RH for small compared to medium MP and when comparing low to standard RH for medium compared to large MP. Finally, there were significant and trend effects of RH on the steering wheel reversal rate of 1° and 10°, , , where the low configuration had a significantly higher steering wheel reversal rate compared to the standard and high, reflecting a higher control activity and less relaxed style of driving in low RH, representing deteriorated performance.
Straight Road Sections
The yaw rate was significantly higher for the low RH as compared to the standard, , . The longitudinal acceleration was significantly higher for the high RH as compared to the low, , . The lateral acceleration was significantly higher for the low RH as compared to the standard, , . A significant interaction effect was also found between the type of MP and the type of RH used on the steering wheel reversal rate 1°, , , when comparing low to standard RH for medium compared to large MP. Finally, the steering wheel reversal rate 10° was significantly higher for the low RH as compared to the high, , showing higher control activity in low RH.
Whole Track
The DPYRE steering gain was significantly higher for the small MP as compared to the medium, , which showed lower control activity and a more relaxed style of driving with the increase of MP size, showing improved performance. The steering wheel reversal rate 1° was significantly higher for the low RH as compared to the high, , . Finally, there was a significant interaction effect between the type of MP and the type of RH used on the steering wheel reversal rate 1°, , , when comparing low to medium MP for small compared to medium and medium compared to large RHs.
Considering the summary of results represented in
Table 10, in the majority of evaluations, the low RH had the highest values in all MPs for all of the objective measures in both straights and curves road sections. These included the steering wheel reversal rates metric that showed higher control activity and less relaxed style of driving in low RH, i.e., deteriorated performance. This consistency indicated the results were independent of the road sections.
3.3. Discussion
Subjectively, the drivers were able to distinguish between the RHs in many (but not all) of the vehicle attributes and rated them correctly. They found standard and low RH more appropriate for the vehicle in terms of the steering and handling qualities of the vehicle. The observed choice of the low RH by the drivers in some of the attributes could be explained by professional drivers’ desire for a racing driving style, which was more eminent in low RH. The preference of the high RH in ride attribute might be because of the drivers’ expectation from driving an SUV style car (to be floatier and more comfortable).
Drivers’ performance was found to be worse in the standard and low RH. In all of the MP sizes and all of the objective measures, the low RH had the highest values. These included the steering wheel reversal rates that show the highest control activity and a less relaxed style of driving in the low RH. Moreover, the longitudinal velocity and acceleration of the vehicle were the highest for the low RH. At low speed, the power steering system decreases the steering effort, and at high speed, it increases the steering effort to dampen any sudden steering input by the driver to improve steering control [
15], which could also explain the higher control activity for low RH. The low RH had the highest lateral acceleration that is directly related to the observed highest lateral position deviation.
The subjective preferences of a configuration and deteriorated objective performance show an inconsistency. In this study, the conventional definitions for the driver performance metrics were used. It considers a lower lateral position deviation and steering control effort as better performance, which is appropriate for normal driving. The racing driving style might hypothetically explain drivers’ higher speed variability, lateral position deviations and steering control efforts. However, these are considered as a deteriorated performance by conventional definitions, which may explain the inconsistency between the subjective preferences and performance and suggests an alternative interpretation might be needed. Strictly speaking, the conventional metrics of driver performance are not well adapted to professional drivers and might not be perfect for assessing expert driver performance.
Another observation was a direct relationship in drivers’ ratings between the MP size and choice of appropriate RH. In the lower RH, the smaller MP was selected as more appropriate, and vice versa. A possible explanation is the lower frequency of the vehicle roll and pitch motions in the high RH, and inversely the higher frequency in the low RH that affects the perception of lateral and longitudinal motions. Usually, the lower frequency vehicle motions require larger MP excursions to generate the motions, and therefore larger longitudinal and lateral motions (low frequencies) may have been expected by the drivers while experiencing the high RH, which is available in the larger MP. This is confirmed because the drivers have subjectively identified that the roll of the vehicle has increased in the higher RH and also identified that the vehicle roll has increased with the increase of MP size.
There were a small number of significant effects of MP on the objective metrics; where the drivers had lower control effort and lateral position deviation in the larger MPs, which means they had a more relaxed style of driving that resulted in a higher lap time. This was aligned with the literature where it has been reported that increasing motion reduces the steering control effort [
33,
34].
4. Conclusions
An experiment was designed and conducted to investigate the feasibility of using driving simulators to study the changes in vehicle ride, steering and handling attributes due to manipulation of ride height (RH) and simulator motion platform (MP) workspace size. Overall, numerous effects of vehicle ride height and motion platform workspace adjustments on the subjective and objective measurement metrics were observed.
The results showed the differences between the RHs were subjectively distinguishable by the drivers in many of the vehicle attributes. Drivers found standard and low RHs more appropriate for the vehicle in terms of the steering and handling qualities, where their performance was deteriorated, such that the steering control effort was the highest in low RH. This suggested inconsistency between subjective preferences and objective performance and the need for alternative performance metrics to be defined for expert drivers. Moreover, an improvement in drivers’ performance was observed, with a reduction of steering control effort, in larger MP configurations. The findings showed that some vehicle design aspects could be tuned using a driving simulator.
The subjective results were comparable with the real-world about the most appropriate RH or MP configuration (since the drivers had the real-world testing experience). However, the analysis of objective results was explorative, and it requires further comparison with real-world data to conclude on the drivers’ best performance in any of the configurations. Moreover, it would be interesting in the future to study the human perception of vehicle properties between normal and professional drivers.
The vehicle design consists of several components, such as body shell, powertrain, transmission system. More research is needed in each of these areas to see if driving simulators are applicable for virtual prototyping. Depending on the aspect of vehicle design, the simulator motion platform requirement varies. For instance, if the aim is to evaluate the vehicle ride attribute on a flat road with undulations, which is perceivable mostly via vertical vibrations, the simulator could have a small translational motion workspace but high bandwidth to capture the high-frequency motions.
Furthermore, since both subjective and objective measurement data are available, it would be an interesting addition to look at the correlations between them to explore indicators for new vehicle design and development, using regression analysis and neural networks [
4,
5]. By extending the experimental design and scenario, it is possible to address other issues, such as drivers’ mental efforts and simulator sickness, while evaluating their perception and behaviour. Transferability of these experiments to other simulators with a range of visual and vestibular characteristics is a worthwhile future research topic.