Usability Evaluation of Co-Pilot Screen Based on Fuzzy Comprehensive Evaluation Method

Abstract

In this study, the usability evaluation model is constructed for a co-pilot screen, and an analysis of the impact factors and optimization recommendations is made based on the evaluation results. Firstly, based on the usability design principles, interaction ease, interaction efficiency, visual comfort, driving safety, and their corresponding secondary indicators are defined, and the subjective weight of each indicator is determined using the analytic hierarchy process (AHP). Then, usability evaluation is carried out on four vehicles via vehicle driving simulated experiments and driving experiments on the road, and the objective weight of the indicators is determined using the CRITIC method. Finally, the usability evaluation model for co-pilot screens is established by applying the fuzzy comprehensive evaluation method. The results indicate that the overall usability comprehensive score of co-pilot screens is convergent and is mainly concentrated in the range of 50–65 points, with two vehicles having excellent affiliation and two vehicles having average affiliation. However, there is a great distance still to reach when compared to an excellent level. The usability evaluation model of co-pilot screens established in this article can quantify the HMI usability design of co-pilot screens. The results of this study are significant for the four tested vehicles in terms of guiding the usability design of co-pilot screens and in promoting the rapid iteration of co-pilot screen development. And a production vehicle that connects a driving simulation platform and the usability evaluation model can be used to test and evaluate more screen designs, interaction models, tasks, and infotainment applications, thus guiding further user experience designs.

1. Introduction

With the boom in the intelligent cockpit industry, users are placing higher demands on in-vehicle information systems; therefore, in the current day, the functions and interaction layouts of the cockpit should not only focus on the driver but should also take into account the needs of the co-pilot. In line with this trend, vehicle manufacturers are launching several vehicles with co-pilot screens.

The co-pilot screen not only changes the layout of the screen in the cockpit, bringing users the most direct sense of technology but also reflects that the new hardware and software architecture can further enhance the user experience. However, due to the low level of maturity, the co-pilot screen has not yet formed relevant design principles and industry standards. The design style varies greatly from vehicle to vehicle, resulting in a wide range of individual experiences. At the same time, the video and audio played on the co-pilot screen may take up the driver’s visual and auditory resources, leading to driving distractions and increasing driving safety hazards.

Currently, the research on cockpit screens has focused on the usability of the human-machine interface (HMI) of the center screen, the HUD, etc. It is important to evaluate the usability of the prototype HMI design before it is fitted into a production vehicle [1]. Yating Su et al. [2] established a research framework, using the literature research method, for the usability of automated vehicle HMIs from criteria such as usability issues and tools. Tinga et al. [3] designed prototype HMIs in different automation modes based on ambient light, icons, LED bar lengths, and timers; this was achieved using virtual reality and the RITE (rapid iterative testing and evaluation) method to evaluate and improve the HMI design, which was verified to have good system usability. Jiajie He [4] selected the ease of operation, operation load, user satisfaction, and other indicators to measure the usability of the HMI of the center screen. Rui Li et al. [5] found that menu architecture impacted the usability of in-vehicle information systems by using the completion time of tasks, error rate, NASA-TLX, and the system usability scale (SUS). In addition, the content of the HMI display can also lead to changes in the usability of the HMI [6].

Keeping driving safe is a fundamental design principle for vehicle usability. Many papers have studied the effect of performing secondary tasks such as touchscreen interaction [7], the use of mobile phones [8], in-vehicle information system engagement [9], etc., on driving distraction. Strayer et al. [10] studied the visual and cognitive demands of vehicle information systems under secondary tasks such as audio entertainment, calling and dialing, text messaging, and navigation. Grahn et al. [11] pointed out that the user interface design has a relatively high impact on visual demand and visual distraction potential compared with screen size. Different types of interaction modes can be used to reduce the effects of driving distraction in tasks of varying complexity and difficulty [12,13]. Driving data and visual data are important objective indicators in the field of HMI and driver distraction research. Driving performance, such as speed deviation, lane departure standard deviation, etc., is commonly combined with the support vector machine (SVM), random forest, or other models to evaluate the degree of driving distraction [14,15,16]. The visual data of drivers are collected via eye trackers, such as areas of interest, glance, gaze, etc., and these are widely used to evaluate the usability of the HMI, determine the state of driving distraction, and guide HMI design in the field of the automotive and aerospace industries [17,18,19]. Furthermore, most of the data were obtained under simulation or a lab environment [20].

As a trend in intelligent cockpit development, the design of co-pilot screens needs to ensure the effectiveness, efficiency, and satisfaction of HMIs while minimizing the impact of driving distractions; thus, it is important to build a comprehensive usability evaluation model of co-pilot screens. However, there are gaps in the research on co-pilot screens in two respects. Firstly, there is no evaluation indicator system for the HMIs of co-pilot screens in terms of assessing the ease of use and efficiency. Secondly, whether the images and sounds of a co-pilot screen can lead to driving distractions and to threatening driving safety remains unstudied. Therefore, in order to solve the research problem, this article first built a multi-layer evaluation indicator system, then calculated the indicator weights using the AHP and CRITIC methods. Finally, the fuzzy comprehensive evaluation method was applied to build a usability evaluation model.

2. Methodology

A comprehensive evaluation indicator system is the core of the usability evaluation model, and this can reflect the advantages and shortcomings of the co-pilot screen from multiple aspects, as well as guide the optimization of co-pilot screens. After the multi-layer evaluation indicators are determined, this article adopted the comprehensive weight method based on the AHP and CRITIC methods to calculate the evaluation indicator weights. This method can reflect the objective data value of each evaluation indicator and can reduce the subjective influence. The calculation steps are shown in Figure 1. The fuzzy comprehensive evaluation is widely used in industrial production evaluation. Qiong Zhang et al. [21] established evaluation indicators based on the principles of comparability, scientificity, unity, and practicability, as well as adopted the fuzzy comprehensive evaluation method to conduct safety evaluations on the whole process of C919 single-test flights. Shuai Wang et al. [22] adopted the fuzzy comprehensive evaluation to evaluate the application effectiveness of the connected vehicle system in tunnel scenarios. Zhijie Zhu et al. [23] developed a prediction method for coal burst that was based on the AHP and fuzzy comprehensive evaluation methods, and this made coal burst prevention more effective. Therefore, this article uses a fuzzy comprehensive evaluation to evaluate the usability of the co-pilot screen.

The final co-pilot screen usability evaluation model is shown in Figure 1, based on which experimental validation of the co-pilot screen usability is carried out.

2.1. Evaluation Indicator System

According to ISO 9241 [24], the usability of the co-pilot screen is defined as the degree of effectiveness, efficiency, and user satisfaction when using the co-pilot screen to complete the specified tasks in the specified scenarios. Considering that the HMI design of the vehicle must be based on the premise of driving safety, the usability design principles of the co-pilot screen are summarized as follows:

• Clear and easy to operate: appropriate layout of visual elements, clear images without error texts, timely provision of adequate feedback and guidance for users, and it facilitates users to perform the target tasks correctly;
• Efficiency: the most concise operation steps, the shortest time to complete the target tasks;
• Visual comfort: optimal screen size and position to reduce the sense of dizziness in the process of completing the target task;
• Safety: low driving distraction and stable driving performance.

Based on the usability design principles of the co-pilot screen, the evaluation indicator system is categorized into four primary indicators: ease of interaction, efficiency of interaction, visual comfort, and driving safety. Then, several secondary indicators are defined, as shown in

Table 1. Usability evaluation indicators for co-pilot screens.

Primary Indicator	Secondary Indicator	Definition
Ease of interaction	The clarity of information display (C1)	Evaluate the clarity of information display based on the brightness and contrast of the screen, the size, the spacing, and the color of the characters and other elements
	The reasonability of information layout (C2)	Evaluate the reasonableness of the information layout based on the principle of user-friendliness
	The user habituation (C3)	Whether the system layout and interaction settings are consistent with the user’s usage habits
	The perceptibility of feedback (C4)	The user perceptibility of system feedback when users are performing target tasks
	The smoothness of operations (C5)	The timeliness of feedback without delay or frustration
Efficiency of interaction	Operation displacements (C6)	Total linear distance between the bottom right corner of the co-pilot screen and the bottom right corner of the corresponding function button
	Operation steps (C7)	The number of steps taken by the user between the start and the end of the target task
	Task elapsed time (C8)	Time taken by the user between the start and the end of the target task
Visual comfort	Visual comfort (C9)	The difference in the user’s visual comfort score before and after performing the target task
Driving safety	Lane keeping (C10)	The standard deviation of lane departure
	Speed keeping (C11)	The standard deviation of vehicle speed departure
	The number of gaze points (C12)	The number of points at which the user’s eyes are directed to the co-pilot screen
	The number of sweeps (C13)	The number of times the user’s eyes drifted to the co-pilot screen
	Average sweep time (C14)	Average time spent by the user in each eye drift
	Reaction time (C15)	Time is taken by the user from hearing the warning tone to pressing the button

.

2.1.1. Ease of Interaction

To facilitate users’ use of the product, Hao Yang et al. [25] argued that HMI interface designs should follow the four principles, including high resolution, clear design features, all elements being included in the image as much as possible, and high similarity between images. Jun Ma et al. [26] comprehensively evaluated ease of use using subjective evaluation indicators such as reasonable information layout and icon readability, as well as objective evaluation indicators such as the operation error rate. Due to the high similarity between the design of the co-pilot screen and the center screen, some habits of using the center screen could be carried over to the co-pilot screen. During the process of completing the target task, the user’s learning cost and the familiarity of such system will also affect the convenience. Therefore, five indicators are chosen to evaluate the ease of interaction: the clarity of information display, the reasonability of information layout, the user habituation, the perceptibility of feedback, and the smoothness of operations.

2.1.2. Efficiency of Interaction

Jinfei Huang [27] used subjective evaluation indicators of the SUS, such as the interface complexity and the operation complexity, to measure efficiency. Shucong Yu et al. [28] evaluated the efficiency in terms of five indicators such as dynamic operation time, the number of operation steps, and the operation displacement. Because the co-pilot screen is currently installed far away from the passenger seat in vehicles, the user has to maintain a forward-leaning and unstable posture when using the co-pilot screen. And the efficiency of completing the target task will be affected by too many operation steps and too scattered interaction layout. Therefore, three indicators are used to evaluate the efficiency of interaction: operation displacements, operation steps, and task elapsed time.

2.1.3. Visual Comfort

While the vehicle is in motion, watching videos or pictures may induce motion sickness due to conflicting human visual and vestibular perceptions [29]. Chengming Chen et al. [30] used a combination of the eye tracker data, the electrocardiogram, and a streamlined version of the simulator sickness questionnaire (SSQ) to investigate the effects of LCD and OLED on visual fatigue. Cai Li et al. [31] conducted a twenty-participant experiment using the SSQ and postural stability tests to verify that the consistency of motion and visual state had a large effect on the degree of motion sickness. In order to measure the user’s viewing experience of the co-pilot screen in a vehicle motion situation, this article uses the SSQ scale to evaluate visual comfort.

2.1.4. Driving Safety

Although the driver does not directly operate the co-pilot screen while the vehicle is in motion, the video and audio played on the co-pilot screen may cause visual and cognitive distraction for the driver, threatening driving safety. Yang Zhou et al. [32] used the vehicle data such as the speed and lane position deviation, longitudinal and lateral acceleration, and the distance to the front vehicle, as well as the steering wheel data such as the steering wheel angle and steering wheel slew rate, combined with eye and head movement indicators. Through this approach, they built a random forest model that can identify different types of distraction. Strayer et al. [33] introduced the detection response task (DRT) to assess the cognitive demand of different information systems by analyzing the reaction time. Huimin Ge et al. [34] comprehensively summarized driving distraction recognition indicators and analyzed their strengths and weaknesses through literature research, which included not only driving performance, eye tracker, and reaction time, but also physiological-psychological indicators such as electrocardiogram, electroencephalogram, and electrodermal. However, the physiological-psychological indicators are greatly influenced by the users’ physical quality status, and the individual data variability is high. Therefore, the standard deviation of lane departure and vehicle speed departure are selected as driving performance indicators. The number of gaze points, number of sweeps, and average sweep time as eye tracker indicators, combined with DRT reaction time indicators are selected to evaluate driving safety.

2.2. Evaluation Indicator Weights

2.2.1. Subjective Weights

The AHP is a subjective assignment method proposed by Saaty, which decomposes complex problems with multiple objectives and elements into objective, criterion, and indicator layers and analyzes them layer by layer [35]. Although the AHP can reflect the importance that experts attach to different evaluation indicators, they are influenced by the degree of information received and the degree of professionalism. The subjective randomness is too strong, so the AHP cannot fully reflect the advantages of each evaluation indicator.

2.2.2. Objective Weights

The CRITIC method is an objective assignment method proposed by Diakoulaki that takes into account the comparative strength and conflicting nature of the data [36]. Although the CRITIC method can reflect the intrinsic value of the data itself, it is also limited in that it relies entirely on the content of the sample data for analysis.

2.2.3. Comprehensive Weights

Xinglai Xu [37] proposed the comprehensive assignment method that combined the AHP with the CRITIC method, which considers the advantages and disadvantages of subjective and objective weights. Therefore, the comprehensive assignment method is chosen for this article, and the calculation formula is shown in Equation (1).

W = αW_AHP + (1 − α) W_CRITIC (0 ≤ α ≤ 1)

(1)

where W is the comprehensive weight, α is the subjective weight preference coefficient, W_AHP is the subjective weight calculated using the AHP, and W_CRITIC is the objective weight calculated using the CRITIC method. By analyzing the relevant literature [26] and the actual situation, α = 0.5 is taken.

2.3. The Fuzzy Comprehensive Evaluation Model

Since fuzzy theory enables decision makers to make interval judgments while considering uncertainty or fuzziness, the fuzzy comprehensive evaluation method combined with the AHP is widely used [38]. Considering the fuzzy boundaries and subordination relationships between indicators at all levels in this article, the fuzzy comprehensive evaluation model is constructed using the subordination function. The formula for calculating the affiliation function is shown in Equation (2).

$$\begin{gathered} r_1\left(c\right)=\left\{\begin{array}{c}1 0\le c\le 0.1\\ \frac{1}{2}\left[\sin \left(\frac{c-0.1}{0.2}+\frac{1}{2}\right)\pi +1\right] 0.1<c<0.3\\ 0 c\ge 0.3\end{array}\right. \\ {r}_2\left(c\right)=\left\{\begin{array}{c}0 0\le c\le 0.1\\ \frac{1}{2}\left[\sin \left(\frac{c-0.3}{0.2}+\frac{1}{2}\right)\pi +1\right] 0.1<c<0.5\\ 0 c\ge 0.5\end{array}\right. \\ {r}_3\left(c\right)=\left\{\begin{array}{c}0 0\le c\le 0.3\\ \frac{1}{2}\left[\sin \left(\frac{c-0.5}{0.2}+\frac{1}{2}\right)\pi +1\right] 0.3<c<0.7\\ 0 c\ge 0.5\end{array}\right. \\ {r}_4\left(c\right)=\left\{\begin{array}{c}0 0\le c\le 0.5\\ \frac{1}{2}\left[\sin \left(\frac{c-0.7}{0.2}+\frac{1}{2}\right)\pi +1\right] 0.5<c<0.9\\ 0 c\ge 0.9\end{array}\right. \\ {r}_5\left(c\right)=\left\{\begin{array}{c}0 0\le c\le 0.7\\ \frac{1}{2}\left[\sin \left(\frac{c-0.9}{0.2}+\frac{1}{2}\right)\pi +1\right] 0.7<c<0.9\\ 1 c\ge 0.9\end{array}\right. \end{gathered}$$

(2)

where r₁ is the indicator affiliation ‘poorer’, r₂ is the indicator affiliation ‘poor’, r₃ is the indicator affiliation ‘normal’, r₄ is the indicator affiliation ‘good’, r₅ is the indicator affiliation ‘excellent’, and c is the indicator value.

3. Usability Experiment of Co-Pilot Screens

3.1. Participants

In this experiment, twelve participants aged between twenty and twenty-six years were selected, with a male-to-female ratio of 1:1. They were required to hold valid driving licenses for at least one year, to be in good health, and to have a normal or corrected-to-normal vision. The eye tracker had corresponding glasses that were provided to participants wearing glasses. The experiment was conducted in accordance with the Declaration of Helsinki and approved by the Science and Technology Ethics Committee of Tongji University (tjdxsr012) for studies involving humans.

3.2. The Design of Experiments

3.2.1. Vehicle Simulated Driving Experiment

The simulated driving experiment can not only collect the required data quickly and accurately but also explore more extreme driving conditions while ensuring driver safety [39]. Čubranić-Dobrodolac et al. [40] used a simulator with three LCD monitors, three connected computers, and a driver cockpit to study driving behaviors. Yujia Liu et al. [41] built a detachable small and medium-sized test rig based on the Unity3D platform and a driving simulator. Jun Ma et al. [26] innovatively built a simulated driving platform consisting of a ring screen, a data acquisition system, and a vehicle, which further improved the rationality of the simulated driving experiment. To ensure the accuracy of the experimental data and take into account the safety and reproducibility of the experiment, this article used a production vehicle connected to the driving simulation platform to collect the driver-oriented driving safety data, as shown in Figure 2.

The platform consisted of a simulated driving environment, a vehicle, and a data acquisition system as follows:

• The driving simulation scenario: a circular three-lane highway experimental route was built based on the SCANeR platform, with multiple high-definition projectors projecting simulated driving scenarios onto the curved screen. The diameter of the screen is six meters, which provides a horizontal field of vision of two hundred and forty degrees.
• The vehicle: the front wheels of the production vehicle were positioned on the steering base, which was connected to the angle sensor to output the vehicle steering signal. The accelerator and brake pedals were equipped with synchronous sensors to output data. The signals from the steering wheel, the accelerator pedal, and the brake pedal were transmitted into the ACQUISITION module of the SCANeR platform to control the host vehicle.
• The data acquisition system: the driving data were collected via a vehicle-mounted angle sensor. The data of an auditory detection response task (DRT) were obtained using a microswitch attached to the left of the steering wheel. The participants were required to wear eye trackers to collect the visual data. The product model of the eye tracker is Tobii Pro Glasses 3 with a sampling frequency of fifty Hz.

3.2.2. Driving Experiments on the Road

Because the frequency use scenario of the co-pilot screen is the operation and viewing of the screen by the passenger during the driving process, and this behavior does not pose a threat to driving safety, this article used driving experiments on the road to collect the data on ease of interaction, efficiency of the interaction, and visual comfort for the passenger, in order to ensure the accuracy and reliability of the data.

3.3. The Experimental Process

The experimental process was divided into three stages as follows:

• The pre-experiment preparation: after checking the participant’s identity information, the tester introduced the background of the experiment, the process, and the devices and asked the participant to complete the adaptation training of the co-pilot screen, the simulated driving platform, the eye tracker, and the DRT device until they could use them proficiently.
• Vehicle simulated driving experiments: on the production vehicle connected driving simulation platform, the tester was positioned in the passenger seat to record the data Then, the co-pilot screen played an audio video, and the participant wore an eye tracker and maintained sixty km/h while completing the DRT task; the duration was two minutes. After the simulated driving was over, the participants answered the questions related to the video played on the co-pilot screen.
• Driving experiments on the road: the tester was positioned in the driver seat at sixty km/h, and the participant opened the video app by clicking on the co-pilot screen, entered the name of the specified video in the search box, searched, and played the video in full screen, then gave the relevant subjective evaluation score; the participant filled in the SSQ before viewing and again after ten minutes of watching the video.

4. Results

In this article, three automotive HMI industry experts, four doctors and postgraduates in the field of driving distraction, and three drivers with five years of driving experience were invited to fill in the AHP questionnaire based on the one-to-nine scale method. The users were asked to compare their scores two by two to obtain the judgment matrix for each level of indicator weights, and the weight vector was obtained based on the matrix. After passing the consistency test, the weight vector W was normalized to obtain the AHP-based primary indicator weight vector W_AHP1 = [20.64% 11.61% 4.82% 62.93%]^T, and the secondary indicator weight vector W_AHP2 = [5.81% 2.85% 1.66% 0.64% 0.64% 2.11% 3.55% 14.98% 4.82% 11.55% 4.68% 11.55% 4.25% 28.85% 2.07%]^T.

In this article, twelve participants were invited to conduct experiments on four vehicles equipped with co-pilot screens, and the test data of fifteen evaluation indicators were obtained in forty-eight sets of experiments. Differences in the data dimension could lead to different results. To eliminate the impact of varying data dimensions and ensure comparability, the data of each evaluation indicator was standardized to make them consistent. The positive indicators were processed by applying Equation (3). Negative indicators were processed by applying Equation (4).

x′ = (x − min(x))/(max(x) − min(x))

(3)

x′ = (max(x) − x)/(max(x) − min(x))

(4)

where x′ is the standardized indicator data, min(x) is the minimum value of the data for the same indicator, and max(x) is the maximum value of the data for the same indicator.

The secondary indicator weights W_CRITIC = [7.71% 6.42% 6.96% 7.16% 7.83% 10.9% 0 6.73% 5.53% 7.37% 7.37% 6.05% 6.13% 7.54% 6.30%]^T, which were calculated using SPSSPRO based on the CRITIC method.

The comprehensive weights of the indicators at each level were calculated based on Equation (1), as shown in

Table 2. The comprehensive weights of the indicators at each level.

Secondary Indicators	Ease of Interaction	Efficiency of Interaction	Visual Comfort	Driving Safety	Comprehensive Weights
	23.84%	19.14%	5.18%	51.85%
The clarity of information display	6.76%	0	0	0	6.76%
The reasonability of information layout	4.64%	0	0	0	4.64%
User habituation	4.31%	0	0	0	4.31%
The perceptibility of feedback	3.90%	0	0	0	3.90%
The smoothness of operations	4.23%	0	0	0	4.23%
Operation displacements	0	6.50%	0	0	6.50%
Operation steps	0	1.78%	0	0	1.78%
Task elapsed time	0	10.86%	0	0	10.86%
Visual comfort	0	0	5.18%	0	5.18%
Lane-keeping	0	0	0	9.46%	9.46%
Speed keeping	0	0	0	6.03%	6.03%
The number of gaze points	0	0	0	8.80%	8.80%
The number of sweeps	0	0	0	5.19%	5.19%
Average sweep time	0	0	0	18.19%	18.19%
Reaction time	0	0	0	4.18%	4.18%

.

From the affiliation function Equation (2), the affiliation degree of indicators at each level was calculated and the fuzzy judgment matrixes for four vehicles (A1, A2, A3, and A4) were obtained, as shown in

Table 3. The fuzzy judgment matrix of the A1.

Indicator	Poorer	Poor	Normal	Good	Excellent
C1	0	0	0	1	0
C2	0	0	0.067	0.933	0
C3	0	0	0	0.500	0.500
C4	0	0	0	0.067	0.933
C5	0	0	0	0	1
C6	1	0	0	0	0
C7	0	0	1	0	0
C8	0	0	0	0.194	0.806
C9	0	0	0.001	0.999	0
C10	0	0	0.013	0.987	0
C11	0	0	0.020	0.980	0
C12	0	0.160	0.840	0	0
C13	0	0.294	0.706	0	0
C14	0	0	0.892	0.108	0
C15	0	0	0.785	0.215	0

,

Table 4. The fuzzy judgment matrix of the A2.

Indicator	Poorer	Poor	Normal	Good	Excellent
C1	0	0	0.933	0.067	0
C2	0	0.067	0.933	0	0
C3	0	0.854	0.146	0	0
C4	0.500	0.500	0	0	0
C5	1	0	0	0	0
C6	0	0	0	0	1
C7	0	0	1	0	0
C8	0.105	0.895	0	0	0
C9	0	0	0.717	0.283	0
C10	0	0	0	0.076	0.924
C11	0	0	0.007	0.993	0
C12	0	0	0	0.979	0.021
C13	0	0	0.000	1.000	0
C14	0	0	0	0.967	0.033
C15	0	0	0.991	0.009	0

,

Table 5. The fuzzy judgment matrix of the A3.

Indicator	Poorer	Poor	Normal	Good	Excellent
C1	0.067	0.933	0	0	0
C2	0	0	0	0.500	0.500
C3	0	0	0.500	0.500	0
C4	0	0	0	0.933	0.067
C5	0	0	0.854	0.146	0
C6	0	0	0.690	0.310	0
C7	0	0	1	0	0
C8	0	0	0.007	0.993	0
C9	0	0.035	0.965	0	0
C10	0	0	0.860	0.140	0
C11	0	0.455	0.545	0	0
C12	0	0	0.957	0.043	0
C13	0	0	0.989	0.011	0
C14	0	0	0.011	0.989	0
C15	0	0.052	0.948	0	0

and

Table 6. The fuzzy judgment matrix of the A4.

Indicator	Poorer	Poor	Normal	Good	Excellent
C1	0	0.067	0.933	0	0
C2	0	0.067	0.933	0	0
C3	0	0.854	0.146	0	0
C4	0	0	0.933	0.067	0
C5	0	0.500	0.500	0	0
C6	0	0	0	0.178	0.822
C7	0	0	1	0	0
C8	0.007	0.993	0	0	0
C9	0	0	0.425	0.575	0
C10	0	0	0.825	0.175	0
C11	0	0	0	0.071	0.929
C12	0	0.168	0.832	0	0
C13	0	0	0.744	0.256	0
C14	0	0	0.979	0.021	0
C15	0	0	0.699	0.301	0

.

Since the comprehensive weights of the indicators at each level were clearly defined, the multiplicative and bounded operator M (*, ⨁), which took into account the magnitude of all factors, was chosen to calculate the usability evaluation vector of the co-pilot screen. To obtain a more intuitive comprehensive usability score, the collection of comments was assigned the corresponding score N = [0 25 50 75 100] and combined with the evaluation vector to calculate the comprehensive score value S. The obtained fuzzy comprehensive evaluation results of co-pilot screens for A1, A2, A3, and A4 are shown in

Table 7. The fuzzy comprehensive evaluation results of co-pilot screens.

Vehicle	Poorer	Poor	Normal	Good	Excellent	Score
A1	0.065	0.029	0.329	0.389	0.188	65.12
A2	0.073	0.156	0.209	0.400	0.160	60.45
A3	0.004	0.094	0.462	0.413	0.026	59.02
A4	0	0.188	0.607	0.094	0.109	53.09

.

5. Discussion

Overall, the comprehensive usability scores for the co-pilot screens of the four vehicles tend to be the same, concentrated in the 50-to-65-point range.

The main affiliation degree of the A1 is 0.389, with an affiliation grade of good. By analyzing the fuzzy judgment matrix of the A1, its affiliation grade is good or excellent in several indicators of ease of interaction, efficiency of interaction, and visual comfort, which indicates that the HMI design of the co-pilot screen of the A1 is reasonable and in line with the user’s usage habits. In several indicators of driving safety, the affiliation grade is normal, with the number of gaze points and the number of sweeps scoring significantly lower than other vehicles. The root of the problem lay in the limited anti-peeping effect of the co-pilot screen film, which only reduced the brightness of the screen from the drivers’ viewpoint but did not completely block the content of the screen. When the video attracted the driver to watch, the driver could not fully access the information of the video played on the co-pilot screen in a short period of time through a single sweep, so the number of gazing points and sweeps increased.

The main affiliation degree of the A2 is 0.400, with an affiliation grade of good. By analyzing the fuzzy judgment matrix of the A2, its affiliation grade is good or excellent in several indicators of driving safety, which is significantly better than other vehicles, indicating that the A2’s co-pilot screen has less influence on driving distraction. The affiliation grade is poor or normal in several indicators of ease of interaction and efficiency of interaction. Interaction design flaws such as small fonts resulted in unclear viewing from a distance. The lack of color, sound, and vibration feedback and the lack of a back button did not match the user’s usage habits. During the participants’ execution of the tasks, the slow response time of the co-pilot screen and the long time taken for the tasks resulted in their low subjective evaluation scores.

The main affiliation degree of the A3 is 0.462, with an affiliation grade of normal. By analyzing the fuzzy judgment matrix of the A3, most of the indicators have normal or good affiliation grades, which indicates that the HMI design of the co-pilot screen is reasonable and has a slight impact on driving distraction. This is due to its low functional content and simple design, with low learning costs for the user. However, due to the small display area of the co-pilot screen, the clarity of the information display is poor.

The main affiliation degree of the A4 is 0.607, with an affiliation grade of normal. By analyzing the fuzzy judgment matrix of the A4, its affiliation grade is good in the indicator of visual comfort. This is probably due to the large display area of its co-pilot screen, which made it easy for participants to quickly capture key information displayed on the co-pilot screen, with a strong sense of audio-visual immersion and a better viewing experience. However, it scored poorly in the task elapsed time of efficiency of interaction because of the lack of fluency of its vehicle information system.

6. Conclusions

This article combines the AHP and the CRITIC method for comprehensive weights, and a usability evaluation model for the co-pilot screen is established based on the fuzzy comprehensive evaluation method. A vehicle driving simulated experiment and driving experiments on the road were carried out with twelve participants. A comprehensive usability evaluation of four vehicles with the co-pilot screen was also carried out, leading to the following conclusions:

• The usability evaluation model of co-pilot screens established in this article can quantify the usability of the HMI design of the co-pilot screen of each vehicle. For the four tested vehicles, there are relatively small differences in the usability of the co-pilot screen, but the overall usability is still far from the excellent level. Compared to the more mature design of the center screen, some of the weaker indicators such as the clarity of information display, the reasonability of information layout, and user habituation still need further improvement.
• The smoothness of the vehicle information system is likely to be a key factor in the smoothness of operation and task elapsed time, which determines the level of ease of interaction and efficiency of interaction on the co-pilot screen of each vehicle.
• The visual comfort aspect remains relatively consistent across the four tested vehicles. And user motion sickness was less affected by the co-pilot screen.
• The driving safety of the co-pilot screen should remain a key concern in the HMI design. A purely visual anti-peep design is not a reliable way to reduce driving distractions. It is recommended to adjust the size of the co-pilot screen and its depth of insertion into the cockpit, combined with headrest speakers or Bluetooth headphones to reduce visual and cognitive distraction.

From the research aspect, this article innovatively selects the co-pilot screen as the study object to provide a new study direction for the HMI field. The production vehicle-connected driving simulation platform and the usability evaluation model can be used to test and evaluate more screen designs, interaction models (voice, gesture control, and buttons), interaction tasks, and infotainment applications to improve the evaluation system of intelligent cockpits. From the practice aspect, the results and optimization recommendations of this study for the four tested vehicles can guide these car companies to update and iterate their co-pilot screens, as well as provide scientific and systematic usability design principles for other car companies to develop new models equipped with co-pilot screens.

Although this article provides some new insights, there are still some research limitations. Firstly, the number of tested vehicles was limited because there were fewer production vehicles equipped with co-pilot screens nowadays. Second, the participants and tested vehicles in this article were all from mainland China, and the conclusions obtained may not be applicable to the global market. Finally, due to the length limitation of this paper, the effects of human segmentation attributes such as the participant’s gender, personality, and driving style on the experimental results were not analyzed in this article.

Future research can further enrich the evaluation indicator system, collect the physiological data such as electrocardiogram, electroencephalogram, heart rate, etc., and combine with interaction tasks such as voice and gesture control to improve the evaluation experiments. In practice, car companies can combine the usability evaluation of multiple screens in the car, including the co-pilot screen, the center screen, and the rear screen, with a large number of user surveys to improve the user experience design.