Objective Evaluation of Motion Cueing Algorithms for Vehicle Driving Simulator Based on Criteria Importance through Intercriteria Correlation (CRITIC) Weight Method Combined with Gray Correlation Analysis

Abstract

Perception-based fidelity evaluation metrics are crucial in driving simulators, as they play a key role in the automatic tuning, assessment, and comparison of motion cueing algorithms. Nevertheless, there is presently no unified and effective evaluation framework for these algorithms. To tackle this challenge, our study initially establishes a model rooted in visual–vestibular interaction and head tilt angle perception systems. We then employ metrics like the Normalized Average Absolute Difference (NAAD), Normalized Pearson Correlation (NPC), and Estimated Delay (ED) to devise an evaluation index system. Furthermore, we use a combined approach incorporating CRITIC and gray relational analysis to ascertain the weights of these indicators. This allows us to consolidate them into a comprehensive evaluation metric that reflects the overall fidelity of motion cueing algorithms. Subjective evaluation experiments validate the reasonableness and efficacy of our proposed Perception Fidelity Evaluation (PFE) method.

1. Introduction

Driving simulators play an increasingly vital role in today’s societal development due to their high safety, reliability, low cost, and diverse training scenarios [1]. However, due to spatial constraints, simulators cannot fully replicate the motion of real vehicles. To address this issue, numerous scholars have long been dedicated to researching and optimizing various motion cueing algorithms, including classical motion cueing algorithm (CMCA) [2], the adaptive motion cueing algorithm (AMCA) [3], the optimal motion cueing algorithm (OMCA) [4], the adaptive washout algorithm-based fuzzy tuning (FMCA) [5], and the model predictive control (MPC)-based MCA [6], among others.

Table 1. Comparison of algorithm characteristics.

Type	Advantages	Disadvantages
CMCA	Fewer parameters, fast calculation.	Fixed parameters, poor applicability.
AMCA	Filter parameters can be adjusted in real time.	The stability is poor, and the optimization efficiency is low.
OMCA	Human perception error is considered for the first time.	The parameters are fixed, and the calculation is complicated.
FMCA	Regulators are plentiful and flexible.	The algorithm structure is complex.
MPC-MCA	Parameters and constraints have a more intuitive relationship.	It is highly dependent on human perception model, which is not perfect.

shows the characteristics of each MCA.

Perception-based evaluation metrics are indispensable in driving simulators, playing a crucial role in the automatic tuning, assessment, and comparison of motion cueing algorithms [7]. However, there is currently a lack of a unified and effective evaluation scheme for motion cueing algorithms. Consequently, numerous researchers have conducted studies on perception-based evaluation metrics.

Felix [8] devised a rating system to evaluate the fidelity of motion simulation by continuously rating 35 participants during simulated driving. While this method provides a straightforward evaluation, its accuracy and reliability are compromised due to various subjective factors influencing participants’ ratings. Casas proposed Motion Fidelity Evaluation (MFE) and Perception Fidelity Evaluation (PFE) [9]. MFE involves comparing the motion signals generated by the simulator with expected motion signals, such as the Objective Motion Cueing Test (OMCT) suggested by Advani and Hosman [10] (see Figure 1). The reliability of motion simulation is assessed by comparing the magnitude and phase differences between the vestibular stimulation signals provided by the motion simulation system (output) and the actual motion signals (input) [11].

Hess [12,13] added the pilot structural model and aircraft dynamics models to the evaluation system (see Figure 2) and put forward a Handling Qualities Sensitivity Function (HQSF) to analyze the similarity between the pilot–aircraft system and the pilot–simulator system from the perspective of frequency domain. The higher the similarity, the better the performance of the MCA.

The advantage of the MFE is that it has definite signal comparison and analysis method and evaluation indicators. While OMCT does not take into account the human perception, which is used to optimize MCA and is important for the MCA evaluation. The model and the determination of the fidelity metric in the Hess method are complicated. And the method has poor universality for other types of driving simulators.

The process of PFE is shown in Figure 3. The input and output signals are denoted as follows:

U₀: motion signal of real vehicle.

U₁: motion signal of reference point of motion platform.

R₀: pose signal of motion platform.

L₀: signal of variation in elongation of each actuator.

P₀: expected perceptual signal.

P₁: simulated perceptual signal.

Firstly, U₀ is calculated by the MCA and inverse kinematics solution to obtain L₀, so as to control the pose of the moving platform, and then obtain R₀. After that, U₀ and U₁ pass through the motion perception model, which is the vestibular model, respectively, to obtain P₀ and P₁. Finally, the comparison between P₀ and P₁ is carried out to complete the evaluation of the MCA [14].

Compared to the MFE, the PFE not only considers perception fidelity but also features a simpler model, making it the most widely used method in the MCA evaluation and tuning. However, there are some deficiencies in this method. Firstly, the mathematical model of motion perception in the PFE falls short of reality. Studies indicate that the human motion perception system mainly comprises the visual system, vestibular system, and somatosensory system. The visual system perceives angular and linear velocity, while the vestibular system, consisting of otoliths and semicircular canals, senses specific force and angular velocity. Zacharias [15] reported that both psychophysical and neurophysiological studies support the theory that visual and vestibular cues are jointly processed to provide a perceived sense of self-motion. Accordingly, Telban [16] proposed a visual–vestibular interaction model. Later, Gustav [17] confirmed that during driving, drivers integrate visual and vestibular cues rather than relying solely on the vestibular system for perceptual information. Tactile or somatosensory receptors sense changes in force on the body, with proprioceptive and kinesthetic senses providing information about the relative position of body parts and their movement [18]. In the study of proprioceptive and kinesthetic senses in humans, Gum [19] developed an initial model that includes muscles, muscle spindles, and the mechanics of head movement. Later, Pham [20] organized and analyzed the model, proposing a model of tilting angle perception of the head. Currently, only the vestibular system model is used in the PFE. Obviously, the perceptual information obtained by the PFE is not perfect.

Secondly, the evaluation result of the PFE is not clear enough, lacking an indicator for the overall performance of the MCA. In advance, P0 and P1, including specific force in transversal, horizontal, and vertical directions, as well as angular velocity in roll, pitch, and yaw, are often compared using curves, as demonstrated in Aminzadeh’s paper [21]. While this approach is relatively straightforward, when comparing and evaluating two or more MCAs, results can be ambiguous due to factors such as the complexity of U0 or small differences between the MCAs. In other words, there is no specific evaluation indicator to clarify the results. To address this issue, some scholars have employed statistical methods to compare and calculate P0 and P1, proposing evaluation indices.

For instance, Mohammad [22,23] and his team compared P0 and P1 by calculating the root mean square error (RMSE) and correlation coefficient (CC) between them. Pham [19] proposed an evaluation index based on the standard error formula. Casas [24] also introduced a set of relevant reference indicators, including Normalized Average Absolute Difference (NAAD), Average Absolute Scale (AAS), Normalized Pearson Correlation (NPC), and Estimated Delay (ED). Higher values of each indicator suggest better MCA performance. These four indicators can be categorized into three types: error analysis indicators, which include NAAD and AAS; correlation analysis indicator, which is NPC; and delay analysis indicator, which is ED. This set of indicators is currently the most comprehensive and is also the focus of this paper.

However, in the existing research, due to the lack of comprehensive indicators during MCA evaluation, the final assessment results remain unclear for a large number of signal types such as perceived forces, angular velocities, and head tilt angles, which need to be compared and evaluated by drivers. To improve the accuracy and efficiency of the PFE, this study adopts a more advanced visual–vestibular interaction model in the evaluation scheme, replacing the vestibular model, and introduces a head tilt angle perception model. The output signals of the improved perception system not only include forces and angular velocities but also linear velocities and head tilt angles, making the perceived information more realistic and enhancing the accuracy and reliability of the assessment results. Additionally, to provide clearer evaluation results, this study selects NAAD, NPC, and ED, combining the aforementioned perception types to establish an evaluation index system. Furthermore, based on CRITIC and combined with gray relational analysis, the weights of each indicator are determined, proposing a comprehensive performance evaluation index for MCAs. This study employs an objective evaluation method for MCAs based on CRITIC weights, combined with gray relational analysis, which holds significant practical value for the assessment and enhancement of MCAs and motion simulator-based evaluations.

The remainder of this study is organized as follows. Section 2 introduces the visual–vestibular interaction model and the head tilt angle perception model, and enhances the perceptual fidelity evaluation scheme. Section 3 establishes an evaluation indicator system and determines the weight value of each index based on the CRITIC method combined with gray correlation analysis. Section 4 designs a subjective evaluation experiment to verify the rationality and accuracy of the proposed PFE in this paper. Finally, Section 5 provides the conclusions.

2. Materials and Methods

2.1. Visual–Vestibular Interaction Model and Evaluation Scheme

2.1.1. Visual–Vestibular Interaction Model

As depicted in Figure 4 (Zacharias, G. 1977), the visual–vestibular interactive model for rotational motion can be utilized to estimate perceived motion for yaw, roll, and pitch stimuli. The model mainly comprises the visual system, semicircular canal, adaptive operator, cosine bell function, visual motion gain, and optokinetic influence.

The parameters in the semicircular canal model are τ₁ = 80 s and τ_a = 5.73 s, respectively. In the visual model, τ_d = 0.09 s. Additionally, τ_w = 5 s and τ_va = 2 s. Based on the model shown in Figure 4 we can obtain the cosine bell function shown in Figure 5. The gain K varies between zero and one. A large conflict signal greater than the conflict threshold ε drives the gain to zero, whereas a small signal below the threshold value drives the gain to a value between zero and one, approaching one as ω_err approaches zero. For ω_err less than zero, the gain remains at one. The conflict threshold ε is chosen to equalize the vestibular indifference motion threshold.

Angular velocities perceived by the visual system and vestibular system are represented as ω_vest and ω_vis, respectively. The visual cue is passed through an internal model of the vestibular dynamics to produce an “expected” vestibular signal, which is then subtracted from the actual vestibular signal. To allow for the long-term resolution of steady-state conflict, the absolute value of e is passed through an adaptation operator to generate a conflict signal ω_err. This signal is then modulated by a visual gain coefficient K using the cosine bell function. The optokinetic influence simulates the impact of the vestibular system perception on the visual system perception.

The visual–vestibular interaction model for translational motion is depicted in Figure 6, which is employed to estimate perceived motion of linear velocity in three directions: transversal, horizontal, and vertical. Its structure is similar to the rotational model, but the difference lies in the replacement of the semicircular canal model with the otolith model. Since the input of acceleration in this model is instantaneous, an integral component needs to be added after the otolith and before the visual system. The parameters in the otolith model are K = 0.4, τ_L = 10 s, τ₁ = 5 s, and τ₂ = 0.016 s, respectively. Additionally, τ_w = 0.2 s and τ_va = 2 s.

2.1.2. Model of Tilting Angle Perception of Head

The model of head tilting angle perception proposed by Pham [19] consists of a dynamic model of the human head and a neck muscle spindle dynamic model. During the rolling motion of the body, the disturbance angular acceleration signal u_h is inputted into the head movement model, where the output signal is defined as the head tilt angle ω_h. Subsequently, the muscle spindle generates the sensation of the head angle y_h. Considering the motion inertia of both the human body and the vehicle, as well as the coordinated tilting motion of the driving simulation platform, the input disturbance angular acceleration in the vehicle for this model is as follows:

$$u_{hv}=-\frac{M_h{a}_{yv}r}{I_h},$$

(1)

where a_yv is the lateral acceleration of the vehicle. In the driving simulation platform, the input disturbance angular acceleration is as follows:

$$u_{hp}={{\phi }^{″}}_p-\frac{M_h{a}_{yp}r}{I_h}+\frac{M_{h}gr{\phi }_p}{I_h},$$

(2)

where φ_p is the roll angle of the platform, and a_yp is the lateral acceleration output of the platform. And M_h = 4.6 kg, r = 0.05 m, I_h = 0.0304 kg/m².

2.1.3. Improved Scheme of MCA Perceptual Fidelity Evaluation

This paper establishes a comprehensive perception system model for humans, which is more advanced than the vestibular model. It includes a visual–vestibular interaction model for rotational and translational motion, a model for perceiving tilting angles of the head, and an otolith model used to perceive specific forces. These models are applied within the PFE scheme. It is important to note that the otolith model mentioned here is the same as the one used in the visual–vestibular interaction model for translational motion. The improved PFE scheme is illustrated in Figure 7.

Since U₁ can only be sensed by the vestibular system and somatosensory system, it is necessary to input U₀ as visual perception signals into the visual–vestibular interaction model. This further underscores the importance of the motion perception system in evaluating the MCAs. The sensing signals generated by the comprehensive perception system not only include specific force and angular velocity but also linear velocity and head tilt angle. Finally, the closer P₀ and P₁ are, the better the performance of the MCA.

2.2. Establishment of the Evaluation Indicator System

2.2.1. Key Performance Indicators

As mentioned above, due to the variety of perception and evaluation indicators, some indicators of PFE may perform well while others may perform poorly. This implies that the overall level of the MCAs cannot be adequately reflected, leaving the final result of the PFE somewhat unclear. Therefore, this paper combines the aforementioned indicators through a weighted sum to create a more sophisticated indicator capable of reflecting the overall performance of the MCAs. Prior to this, a system of indicators should be established, as depicted in Figure 8 and Figure 9. The system is divided into three levels.

The formulation of each indicator at the first level is as follows [12]:

$$MAAD(x,y)=1+\frac{1}{n\cdot \left|x_p\right|}\cdot {\displaystyle \sum _{i=1}^n\left|x_i-y_i\right|},$$

(3)

where x and y represent the data of P₀ and P₁, respectively. x_p represents the peak value of the x signal.

$$NPC(x,y)=\frac{1}{1+PC(x,y)}+\frac{1}{2},$$

(4)

with

$$PC(x,y)=\frac{{\displaystyle {\sum }_{i=1}^n(x_i-\overline{x})\cdot (y_i-\overline{y})}}{\sqrt{{\displaystyle {\sum }_{i=1}^n{(x_i-\overline{x})}^2}}\cdot \sqrt{{\displaystyle {\sum }_{i=1}^n{(y_i-\overline{y})}^2}}},$$

(5)

$$ED(x,y)=1+\partial (x,y)$$

(6)

where ∂(x,y) represents the offset that needs to be applied to the input signal of the MCA to maximize the Pearson Correlation with respect to the motion generated by the motion platform.

Figure 8 includes the overall performance indicator and indicators at the first level. Indicators of the second and third levels under NAAD are shown in Figure 9, where R, P, and Y represent roll, pitch, and yaw, respectively. T, H, and V denote transversal, horizontal, and vertical directions, respectively. The indicator sub-system of NPC and ED is similar to that shown in Figure 9. Subsequently, this paper will determine the weights of each indicator in the entire system.

2.2.2. Weights of Indicators at the First Level

Casas [23] analyzed the correlation between NAAD, NPC, and ED and participants’ perception in the subjective experiment by combining subjective evaluation with objective evaluation and calculated the correlation coefficient. The higher the correlation between an indicator and participants’ perception, the more the indicator can reflect the performance of the MCA in the objective evaluation, and the greater the weight value allocated in the indicator system. Therefore, the weight values of the three indicators at the first level are obtained by normalizing these correlation coefficients, as shown in

Table 2. Weights of each indicator at the first level.

Indicator	Correlation	Coefficient	Average	Weight
NAAD	0.123	0.218	0.1705	0.17
NPC	0.425	0.495	0.460	0.47
ED	0.347	0.357	0.352	0.36

.

Regarding the determination method of the weight values for all indicators in the second and third levels, in consideration of the objectivity of the PFE, this paper adopts a combination of CRITIC and gray correlation analysis.

2.2.3. Criteria Importance through Intercriteria Correlation (CRITIC)

The objective weight of each indicator in the CRITIC method is calculated based on the information contained in the indicator data. The amount of information is expressed by the standard deviation and correlation coefficient between indicators. Standard deviation represents the variance among indicators, while correlation represents the interrelationship between indicators. As an enhancement of the entropy weight method, it effectively captures the differences and conflicts among indicators and demonstrates strong practical utility. The calculation steps for determining the weights of the proposed indicator system based on CRITIC are listed [25].

Step 1: Suppose there are n evaluation objects and m evaluation indicators. Since the units of each indicator differ, the dimensional aspect of the indicators should be normalized prior to processing. The formulas for standardizing positive and negative indicators are as follows:

$$x_{ij}^{\prime }=\frac{x_{ij}-\min \left\{x_{ij}\right\}}{\max \left\{x_{ij}\right\}-\min \left\{x_{ij}\right\}},\mathrm{for}\mathrm{positive}\mathrm{indicators},$$

(7)

$$x_{ij}^{\prime }=\frac{\max \left\{x_{ij}\right\}-x_{ij}}{\max \left\{x_{ij}\right\}-\min \left\{x_{ij}\right\}},\mathrm{for}\mathrm{negative}\mathrm{indicators},$$

(8)

Step 2: Determine the indicator standard deviations using Equation (9):

$${\sigma }_j=\sqrt{\frac{{\displaystyle {\sum }_{i=1}^n{(x_{ij}^{\prime }-{\overline{x}}_j)}^2}}{n}}$$

(9)

where

$${\overline{x}}_j=\frac{{\displaystyle {\sum }_{i=1}^n{x}_{ij}^{\prime }}}{n}$$

(10)

Step 3: Calculate the correlation coefficient $r_{jh}$ between indicator pairs:

$$r_{jh}=\frac{{\displaystyle {\sum }_{i=1}^n(x_{ij}^{\prime }-{\overline{x}}_j)}\cdot (x_{ij}^{\prime }-{\overline{x}}_h)}{\sqrt{{{\displaystyle {\sum }_{i=1}^n(x_{ij}^{\prime }-{\overline{x}}_j)}}^2}\cdot \sqrt{{{\displaystyle {\sum }_{i=1}^n(x_{ij}^{\prime }-{\overline{x}}_h)}}^2}}$$

(11)

Step 4: Determine the quantity of information of each indicator as follows:

$$C_j={\sigma }_j{\displaystyle \sum _{h=1}^n(1-r_{jh})}$$

(12)

Step 5: Calculate the objective weight of each indicator:

$$W_j=\frac{C_j}{{\displaystyle {\sum }_{j=1}^m{C}_j}}$$

(13)

2.2.4. The Method of Gray Relational Analysis

The gray relational analysis method is an essential component of gray system theory. It compares the correlation among factors based on the similarity of each sequence and the optimal sequence, thus reflecting the relative importance of evaluation attributes in the evaluation sequence to a certain extent. Its advantages lie in its lack of requirements for the number of evaluation attributes and evaluation objects, its simplicity in calculation, its high precision in calculation results, and its wide applicability in multi-attribute evaluation and multi-index comprehensive evaluation in uncertain environments. Below is the basic process of the gray correlation analysis method [26].

The optimal set of each indicator is Y₀, that is, Y₀ = (Y_01, Y_02, ⋯, Y_0n), and the corresponding actual data are as follows:

$$Y=\left[\begin{array}{cccc}{y}_{11}& y_{12}& \cdots & y_{1n}\\ y_{21}& y_{22}& \cdots & y_{2n}\\ ⋮& ⋮& ⋮& ⋮\\ y_{t1}& y_{t2}& \cdots & y_{tn}\end{array}\right]$$

(14)

where y_ij represents the corresponding data of each indicator.

Standardize the optimal set and original data according to Equation (15), and obtain the following:

$$M_{ij}=\frac{Y_{ij}}{Y_{0j}}$$

(15)

$$M=\left[\begin{array}{cccc}{m}_{11}& m_{12}& \cdots & m_{1n}\\ m_{21}& m_{22}& \cdots & m_{2n}\\ ⋮& ⋮& ⋮& ⋮\\ m_{t1}& m_{t2}& \cdots & m_{tn}\end{array}\right]$$

(16)

$$M_0=\left[\begin{array}{cccc}{m}_{01}& m_{02}& \cdots & m_{0n}\end{array}\right]$$

(17)

The correlation coefficients of the corresponding elements of each comparison sequence and reference sequence are calculated, respectively, and the formula is as follows:

$${\xi }_{ij}=\frac{\underset{i}{\min } \underset{j}{\min }\left|M_{0i}-M_{ij}\right|+\rho \underset{i}{\max } \underset{j}{\max }\left|M_{0i}-M_{ij}\right|}{\left|M_{0i}-M_{ij}\right|+\rho \underset{i}{\max } \underset{j}{\max }\left|M_{0i}-M_{ij}\right|}$$

(18)

where ρ is the distinguishing coefficient, usually ρ = 0.5.

The correlation degree of each indicator is calculated by Equation (19):

$${\tau }_j=\frac{1}{m}{\displaystyle \sum _{i=1}^m{\xi }_{ij}}$$

(19)

According to the correlation degree, the weight of each indicator is as follows:

$$W_i=\frac{{\tau }_i}{{\displaystyle \sum _{j=1}^n{\tau }_j}}$$

(20)

2.2.5. Determine Combined Weights

Once the comprehensive performance evaluation index system for sports prompting algorithms is determined, the quantitative evaluation of the algorithm’s performance first requires the weights of the evaluation indicators to be determined. The methods for determining the weights of evaluation indicators can be divided into subjective weighting and objective weighting. Subjective weighting involves experts directly judging the weights of indicators based on their experience, or experienced personnel scoring the indicators according to certain criteria, and then calculating the weights using certain methods. Objective weighting is a method of analyzing and processing the actual attribute values of evaluation indicators to obtain the weights. Common objective weighting methods include the coefficient of variation method, CRITIC method, principal component analysis method, entropy method, and gray relational analysis method. The entropy method in objective weighting cannot directly obtain the weights of indicators based on the volatility or correlation of data; thus, it is not suitable for the objective evaluation of sports prompting algorithms; although the coefficient of variation method considers the volatility of data and the principal component analysis method considers the correlation between data, they still lack comprehensive consideration.

Combining gray relational analysis with CRITIC allows for a comprehensive and objective consideration of multiple evaluation factors. Firstly, gray relational analysis determines the correlation between factors, and then, CRITIC is used to determine the weights of each factor, thereby ensuring a more comprehensive decision-making process. Moreover, CRITIC objectively determines the weights of each factor, while gray relational analysis helps reduce the influence of subjective factors on the correlation, further enhancing the objectivity of the decision-making process.

In the CRITIC method, the objective weight of each indicator is calculated based on the information contained in the indicator data. Information is represented by the standard deviation and correlation coefficient between indicators. In contrast, the gray relational analysis determines the weights by comparing the correlation between each indicator [27].

In order to leverage the advantages of both the CRITIC and gray correlation analysis methods to enhance the accuracy of weight values, this paper integrates the weight values obtained from both methods. The weights obtained from the CRITIC and gray correlation analysis methods are denoted as W₁ = (w₁₁, w_12, ⋯, w_1t) and W₂ = (w₂₁, w_22, ⋯, w_2t), respectively. Then, the combined weight can be defined as follows:

$$W_i=\frac{{\tau }_i}{{\displaystyle \sum _{j=1}^n{\tau }_j}}$$

(21)

where α + β = 1, and

$$\left\{\begin{array}{c}\alpha =\frac{{\displaystyle {\sum }_{i=1}^n{\alpha }_i}}{{\displaystyle {\sum }_{i=1}^n{\alpha }_i}+{\displaystyle {\sum }_{i=1}^n{\beta }_i}}\\ \beta =\frac{{\displaystyle {\sum }_{i=1}^n{\beta }_i}}{{\displaystyle {\sum }_{i=1}^n{\alpha }_i}+{\displaystyle {\sum }_{i=1}^n{\beta }_i}}\end{array}\right.$$

(22)

with

$$\left\{\begin{array}{c}{\alpha }_i=\frac{E(w_{1i})}{E(w_{1i})+E(w_{2i})}\\ {\beta }_i=\frac{E(w_{2i})}{E(w_{1i})+E(w_{2i})}\end{array}\right.$$

(23)

Finally, this paper evaluates 15 MCAs according to the process shown in Figure 7. The input signal U₀ remains the same in each evaluation process. It utilizes the evaluation results combined with the aforementioned weight calculation method to compute the weights of indicators at the second and third levels of the indicator system. The final results are presented in

Table 3. Weights of indicators of ED sub-system.

Second Level	CRITIC Weight	GRA Weight	Combined Weight	Third Level	CRITIC Weight	GRA Weight	Combined Weight
Angular Velocity	0.3784	0.2277	0.3031	R	0.2858	0.4074	0.3466
				P	0.2826	0.2831	0.2829
				Y	0.4315	0.3095	0.3705
Specific Force	0.0591	0.3415	0.2003	H	0.5138	0.2867	0.4002
				T	0.2768	0.2873	0.2821
				V	0.2095	0.4260	0.3177
Linear Velocity	0.2775	0.2414	0.2594	H	0.2627	0.3333	0.2980
				T	0.3208	0.2760	0.2984
				V	0.4166	0.3907	0.4036
Head Angle	0.2850	0.1894	0.2372

,

Table 4. Weights of indicators of NAAD sub-system.

Second Level	CRITIC Weight	GRA Weight	Combined Weight	Third Level	CRITIC Weight	GRA Weight	Combined Weight
Angular Velocity	0.3127	0.2330	0.2729	R	0.7947	0.3048	0.5498
				P	0.0931	0.3603	0.2267
				Y	0.1122	0.3349	0.2235
Specific Force	0.4044	0.2600	0.3322	H	0.9067	0.2877	0.5972
				T	0.0501	0.3439	0.1969
				V	0.0433	0.3685	0.2059
Linear Velocity	0.1626	0.2791	0.2209	H	0.6965	0.3262	0.5114
				T	0.2202	0.2980	0.2591
				V	0.0833	0.3758	0.2295
Head Angle	0.1203	0.2278	0.1740

and

Table 5. Weights of indicators of NPC sub-system.

Second Level	CRITIC Weight	GRA Weight	Combined Weight	Third Level	CRITIC Weight	GRA Weight	Combined Weight
Angular Velocity	0.0974	0.2572	0.1774	R	0.5154	0.3067	0.4110
				P	0.3879	0.2904	0.3392
				Y	0.0967	0.4030	0.2498
Specific Force	0.2400	0.2756	0.2578	H	0.7089	0.2977	0.5033
				T	0.2549	0.3135	0.2842
				V	0.0363	0.3888	0.2125
Linear Velocity	0.2092	0.2737	0.2414	H	0.2094	0.3833	0.2963
				T	0.0667	0.3744	0.2206
				V	0.7239	0.2423	0.4831
Head Angle	0.4535	0.1934	0.3234

, where the GRA represents the gray relational analysis.

2.3. Example Analysis

In order to illustrate the detailed process of the objective evaluation method for the MCAs mentioned in this paper, five CMCAs (a, b, c, d, and e) with different parameters are selected as the evaluation objects. Figure 10 displays the vehicle motion signal U₀ input in each evaluation process, encompassing the acceleration signal and angular velocity signal, which are collected from the Unity simulation scene. Figure 11 and Figure 12 show the comparison between sensed specific force, sensed angular velocity, sensed linear velocity, and sensed head tilt angle obtained by these five CMCAs and actual perception. Figure 13 shows the calculation results of various perception data using NAAD, NPC, and ED, respectively, where ED_Wx, ED_Fx, ED_Vx, and ED_Hta, respectively, represent the Estimated Delay of sensed rolling angular velocity, sensed specific force, sensed linear velocity, and sensed head tilt angle. Similarly, other abbreviations in the diagram follow the same pattern. It can be observed that the overall performance of each CMCA cannot be accurately judged due to a large number of perception types and evaluation indexes, which is also the main problem to be addressed in this paper. The comprehensive performance of the five CMCAs is depicted in Figure 13d.

3. Results

To validate the rationality of the PFE and the accuracy of the overall performance index of the MCAs proposed in this paper, a subjective evaluation experiment was designed. The results of this subjective evaluation experiment are then compared with the objective evaluation results.

3.1. Apparatus and the MCAs

The prototype of the motion simulation platform used in this experiment is depicted in Figure 14. The entire set of equipment mainly comprises a visual system, motion system, control system, and driving operating system. The visual system consists of three 32-inch LCD monitors, and a simulation scene based on engineering vehicle kinematics is created using Unity3D software 2021, as illustrated in Figure 15. The scene features various road conditions such as inclines, bends, and straight lanes.

The motion system is a 6-DOF parallel mechanism consisting of an upper platform, a lower platform, six moving branches, and hook hinges. The moving branches are actuated by electric cylinders. The control system comprises a Speedgoat controller master station and a Delta server slave station. The driving operating system includes a game steering wheel, accelerator, and brake pedals. Additionally, the five CMCAs utilized in this experiment are identical to those employed in the example analysis.

3.2. Experimental Procedure

There are a total of 10 participants, including 3 females and 7 males in the experiment. They all had driving experience in ordinary cars, but no or limited experience in driving simulators.

Firstly, one of the five CMCAs was randomly selected, and each participant was allowed to operate freely on the motion simulation platform for 5 min to familiarize themselves with the operation process.

Then, we commenced the actual trial. Each participant was instructed to follow the same route in the virtual scenario and was given a time limit of 10 min. Upon completion of the operation, participants were asked to rate their experience, with 0 being the lowest and 10 being the highest. A higher score indicates a more realistic experience and better performance of the CMCA. The same participants repeated this process until they completed the experience and evaluation for all five CMCAs.

Finally, the average of the 10 scores for each CMCA was calculated to derive its final subjective evaluation result, as illustrated in

Table 6. Results of subjective evaluation.

	CMCA (a)	CMCA (b)	CMCA (c)	CMCA (d)	CMCA (e)
Participant 1	7	6.5	7.5	8	8
Participant 2	6	5	6	7	8
Participant 3	6	5	7	8	9
Participant 4	5	6.5	7	8	8.5
Participant 5	4	7	5	6	7
Participant 6	4	6	5	5.5	6
Participant 7	5.5	6	6	7	8
Participant 8	6	6.5	7	8	8.5
Participant 9	5	5.5	7	6	7.5
Participant 10	6	8.5	6.5	7	7.5
The average	5.45	6.25	6.40	7.05	7.80

.

4. Discussion

Based on the objective evaluation (Figure 13) and subjective evaluation (Table 6), it can be observed that, in the longitudinal, lateral, vertical, pitch, roll, and yaw channels, the NPC index generally follows a similar trend as the subjective evaluation. In the pitch, roll, and yaw directions, the ED index also aligns with the subjective evaluation trend. The overall trend of the objective and subjective evaluations is consistent.

The correlation coefficient ρ between objective evaluation results and subjective evaluation results is calculated. When the objective evaluation index value and subjective evaluation score are closer to 1 and higher, respectively, it indicates better overall performance of the MCA. Therefore, the closer the correlation coefficient is to −1, the more consistent the two evaluation results are. As shown in

Table 7. Normalization of evaluation results of the CMCAs.

The MCAs	CMCA (a)	CMCA (b)	CMCA (c)	CMCA (d)	CMCA (e)
Subjective evaluation results	0.165	0.190	0.194	0.214	0.237
Objective evaluation results	0.207	0.202	0.200	0.199	0.192

, the objective evaluation results and subjective evaluation results are normalized, respectively. The value of ρ = −0.9786 indicates that the PFE and the overall performance index of the MCA proposed in this paper are reasonable and effective.

5. Conclusions

The paper proposes an objective evaluation method for the perception fidelity of MCAs based on a visual–vestibular interaction system and head tilt angle perception system model. Compared with the existing PFE method, it not only considers acceleration and angular velocity perception but also integrates linear velocity perception and tilt angle perception, effectively improving the accuracy and reliability of the evaluation results. Experimental results demonstrate that human drivers’ responses to the MCA are generally consistent with the NPC index in the longitudinal, lateral, vertical, pitch, roll, and yaw channels. In the pitch, roll, and yaw directions, the ED index trend is also generally consistent with subjective evaluation. Furthermore, the overall performance of MCA perception fidelity, as reflected by the combination of CRITIC and gray correlation analysis, aligns with the trend of human drivers’ responses to the MCA.

Certainly, the proposed evaluation indicator system and overall performance indicators in this article are primarily designed for MCAs in automotive simulators. The objective weight of each indicator is determined by the information contained in the indicator data (standard deviation and correlation coefficient). The indicators listed in the article are commonly used indicators in the current driving simulators. Further analysis is needed for future new evaluation indicators to obtain better overall performance indicators.

Future work will further research and validate objective evaluation methods for the motion fidelity of different types of vehicle simulators as well as different types (flight, maritime) of driving simulators.