Research on the Quantitative Method of Cognitive Loading in a Virtual Reality System

: Aimed at the problem of how to objectively obtain the threshold of a user’s cognitive load in a virtual reality interactive system, a method for user cognitive load quantiﬁcation based on an eye movement experiment is proposed. Eye movement data were collected in the virtual reality interaction process by using an eye movement instrument. Taking the number of ﬁxation points, the average ﬁxation duration, the average saccade length, and the number of the ﬁrst mouse clicking ﬁxation points as the independent variables, and the number of backward-looking times and the value of user cognitive load as the dependent variables, a cognitive load evaluation model was established based on the probabilistic neural network. The model was validated by using eye movement data and subjective cognitive load data. The results show that the absolute error and relative mean square error were 6.52%–16.01% and 6.64%–23.21%, respectively. Therefore, the model is feasible.


Introduction
A cognitive load is the ratio of task complexity to the cognitive ability required by the user to complete the task, which can be described as the limited capacity of working memory and attention [1].The cognitive load has a tremendous impact on the user's ability to execute tasks, which is an important humanistic factor directly related to the efficiency of the system operation, job safety, and production efficiency in different fields [2].In the in-vehicle information system (IVIS), the complex and indiscriminate provision of multiple large sets of data may trigger the cognitive load of drivers, resulting in operational errors and traffic accidents [3].Therefore, researchers have been conducting quantitative research on the cognitive load, mainly measuring the working memory capacity and selective attention mechanism changes in two stages [1,2,4,5].Physiological signals (such as heart rate and respiratory rate), brain activity, blood pressure, skin electrical response, pupil diameter, blinking, and gaze are considered biomarkers for quantifying the cognitive load [6,7].There is an information structure that can effectively quantify the cognitive load in Web browsing and Web shopping, minimize the user's information browsing time, or define the optimal point in time to guide the purchase [8].
Differences in individual cognitive ability and how to enhance the cognitive load affect human cognitive control, which leads to different discoveries of physiological changes as the cognitive load [9], and eye movement technology can objectively measure the cognition of users [10].The pupil measure is the cognitive activity index (ICA), which assesses the association between expected eye movements and immediate cognitive load [11,12].The analysis of eye tracking data provides quantitative evidence for the change of the interface layout and its effect on the user's understanding and cognitive load [13].Many researchers use eye movement behavior data [14][15][16] to obtain the user's behavior habits and interest difference to judge the user's cognitive load.Among them, Asan et al. [17] studied the physiological index associated with the eye movement tracking technology and cognitive load.These

Multi-Channel Interactive Information Integration in the VR System
To solve the problem that it is difficult to quantify the cognitive load of users in a virtual reality interactive system, in order to reduce the difficulty of interactive cognitive analysis, some researchers have constructed a multi-modal cognitive processing model that integrates touch, hearing, and vision [24].In order to improve the naturalness and efficiency of interaction, some researchers have also established a multi-modal conceptual model and a system model of human-computer interaction based on the elements of human-computer interaction in command and control [25].By simulating the process of human brain cognition, this paper studies the interactive behavior of a virtual reality system from cognitive and computational perspectives, and then constructs the interactive information integration model of virtual reality, and the final output value is the cognitive load value of users, such that the cognitive load can be quantified.As shown in Figure 1, in order to realize the functions in the interactive system, users use visual, auditory, and other cognitive channels to analyze the task, and eye movement is studied to collect the user's eye movement behaviors under single-channel, double-channel, and triple-channel conditions.The user's cognitive load in the virtual reality system can then be quantified.

Construction of Cognitive Load Quality Evaluation Model
The evaluation model is generally composed of three layers: the first layer is the basic layer, that is, the evaluation quality characteristics; the second layer is the middle layer, which is further explanation of the first layer, that is, the characteristics of the mass quantum; and the third layer is the measurement index.Based on the hierarchical partition theory of the quality evaluation model, this paper analyzes the attributes of a virtual reality interactive system, takes the size of user cognitive load as the quality characteristics of a virtual reality interactive system quality evaluation model, deduces the quality sub-characteristics, and finally establishes the cognitive load quality evaluation model of the virtual reality interactive system with the eye movement technical index as the measurement index, as shown in Figure 2.

Physiological Index of Cognitive Load Based on an Eye Movement Experiment
In an eye movement experiment as a method of implicitly obtaining cognitive load, the visual behavior recorded by the eye movement instrument is more intuitive than the operating behavior for reflecting the cognitive awareness of users.As the most widely used cognitive load assessment method, eye movement technology is mainly based on the number of fixation points, average fixation duration, average saccade length, number of fixation points at the first mouse click, number of backward-looking times, and other experimental data [26] in order to objectively and scientifically evaluate the cognitive load of a virtual reality interactive system.Therefore, this paper chooses eye movement technology as the experimental approach to establish the cognitive load evaluation model based on the probabilistic neural network.

Number of Fixations
The number of fixation points is proportional to the cognitive load of the virtual reality intersection system.The greater the number of fixation points, the larger the cognitive load is, and vice versa [27,28].Therefore, the number of fixation points is introduced as a physiological index to measure the cognitive load of households.

Physiological Index of Cognitive Load Based on an Eye Movement Experiment
In an eye movement experiment as a method of implicitly obtaining cognitive load, the visual behavior recorded by the eye movement instrument is more intuitive than the operating behavior for reflecting the cognitive awareness of users.As the most widely used cognitive load assessment method, eye movement technology is mainly based on the number of fixation points, average fixation duration, average saccade length, number of fixation points at the first mouse click, number of backward-looking times, and other experimental data [26] in order to objectively and scientifically evaluate the cognitive load of a virtual reality interactive system.Therefore, this paper chooses eye movement technology as the experimental approach to establish the cognitive load evaluation model based on the probabilistic neural network.

Number of Fixations
The number of fixation points is proportional to the cognitive load of the virtual reality intersection system.The greater the number of fixation points, the larger the cognitive load is, and vice versa [27,28].Therefore, the number of fixation points is introduced as a physiological index to measure the cognitive load of households.

Physiological Index of Cognitive Load Based on an Eye Movement Experiment
In an eye movement experiment as a method of implicitly obtaining cognitive load, the visual behavior recorded by the eye movement instrument is more intuitive than the operating behavior for reflecting the cognitive awareness of users.As the most widely used cognitive load assessment method, eye movement technology is mainly based on the number of fixation points, average fixation duration, average saccade length, number of fixation points at the first mouse click, number of backward-looking times, and other experimental data [26] in order to objectively and scientifically evaluate the cognitive load of a virtual reality interactive system.Therefore, this paper chooses eye movement technology as the experimental approach to establish the cognitive load evaluation model based on the probabilistic neural network.

Number of Fixations
The number of fixation points is proportional to the cognitive load of the virtual reality intersection system.The greater the number of fixation points, the larger the cognitive load is, and vice versa [27,28].Therefore, the number of fixation points is introduced as a physiological index to measure the cognitive load of households.

Mean Fixation Duration
The more information you carry, the longer your eyes stay fixed, and the more cognitive load you have.To some extent, this evaluation index can reflect the cognitive load of users intuitively [28][29][30].For this reason, the average fixation duration is used as a physiological index to evaluate the cognitive load of users.

Average Pan Length
Scanning length is used to calculate the length of the bevel according to the coordinates of the fixation point, which is mainly used to analyze the path [31,32] to be scanned, and thus to analyze the size of the cognitive load of the user.

The Number of Fixation Points at the First Mouse Click
Before the first mouse click, the greater the number of the user's fixation points, the higher the user's recognition degree, and the smaller the user's cognitive load [33,34].This index is inversely proportional to the cognitive load.

Number of Back Views
The number of backward-looking views represents the cognitive impairment of the user [35].The causes of backward-looking include: (1) cognitive bias of the subjects and (2) a big contrast between the cognitive object and the subjects' mental image symbols.Users need to recognize them repeatedly to establish and construct new mental image symbols.

Cognitive Load Evaluation Model Based on the Probabilistic Neural Network
Theorem 1.The user's cognitive domain is represented by U, and the cognitive domain is composed of cognitive channels C, expressed as: where C α , C β , C λ • • • each represent a kind of cognitive channel, and the cognitive behavior set of users under the comprehensive effect of each cognitive channel is represented as B.Then, the set of cognitive behaviors of the user is: where b i is the index of the user's cognitive behavior, 0 < i < s.
Taking the eye movement characteristic parameters in the virtual reality interactive system as the input layer and the cognitive load as the output layer, a cognitive load quantification model is constructed, as shown in Figure 3.

•
Input layer: This refers to eye movement data of the entire virtual reality tunnel rescue mission, such as the number of fixation points, in a single vision channel, dual vision-audio channel, dual vision-tactile channel, and three visual-audio-tactile channels.It also includes average gaze duration, average squint length, number of gaze points to the first mouse click, number of gaze times, etc.

•
Fusion layer: This refers to incorporating the acquired data into the cognitive load quantification model based on the probabilistic neural network for data collation.

•
Output layer: This refers to the value of the final output after the data fusion processing, which is the value of the cognitive load quantified by the tester under a certain conditional channel.There are y scheme values and s eye movement indicators.The matrix of the eye movement indicator data of each scheme is as follows: The eye movement index matrix is . Each column of the matrix represents eye movement indicator data, and each row represents a test value.As the units of each indicator data are different, it is difficult to directly compare the data, so it is necessary to normalize the data of each column, perform linear transformation of the original data, and map the result value to      0 1 .If the cognitive load value increases with the increase of each set of indicator data, the transfer function is as follows: Conversely, the conversion function is where Max is the maximum value of the indicator data, Min is the minimum value of the indicator data,   p y s , and the improved matrix When , Z is the dimensional column vector of y .The goal of this paper is to find an estimation function, , such that the mean square error represented by: (5) There are y scheme values and s eye movement indicators.The matrix of the eye movement indicator data of each scheme is as follows: The Conversely, the conversion function is where max is the maximum value of the indicator data, min is the minimum value of the indicator data, p = y * s, and the improved matrix B = b ij y×s is: , Z is the dimensional column vector of y.The goal of this paper is to find an estimation function, Z = Z(b), such that the mean square error represented by: Information 2019, 10, 170 is minimized.For a given set of column vectors According to the conditional expectation, the estimated function is: where f (B, Z) is the joint probability distribution function of (B, Z).The estimate for f (B, Z) is: where σ is the smoothing parameter; s is the dimension of B, that is, s kinds of eye movement index parameters are selected; and y is the number of samples, that is, the number of schemes.Then: where the physical meaning of D i is the distance from each input eye movement index to the sample point i, which is the Euclidean distance.Here, σ = max{D i| i=1,2,••• ,y } √ y . Substituting f (B, Z) for f (B, Z) in Equation ( 6), substituting in Equation ( 8), and exchanging the order of summation and integral number, this can be simplified to obtain: The data is then normalized so that the cognitive load value is in the range of [0 − 1], and the normalized processing function is as follows: where CI is the final output, the cognitive load value, of which E = [1 1 1 1 1] and p = y * s.

Evaluation Index
The experimental output error is defined as: where k denotes the number of cognitive channels, CI k * denotes the number of subjective scores for the cognitive load of the virtual reality interactive system under k cognitive channels, and CI k denotes the value calculated by the user cognitive load evaluation model under k cognitive channels.
In this paper, the maximum absolute error ER 1 and the relative mean square error ER 2 are used to evaluate the evaluation effect of the model, and the calculation method is as follows: where H is the total number of channel classes.

Experimental Design
A VR tunnel emergency rescue system mainly obtains rescue information using a visual reading; the auditory system acquires tunnel rescue information, such as tunnel wind sound, water drops sounds, etc., and obtains rescue information; and the touch sense is initiated by touching the handle to obtain the selected rescue information.This paper is mainly focused on the virtual reality system.The tester wore virtual reality equipment and eye-moving equipment; completed the selection of vehicles by visual, auditory, and tactile systems; selected rescue teams; detected life; opened life channels; and provided rescue channels and other rescues.Based on the VR tunnel emergency rescue system, the main focus was on vision.If the experiment was not completed without the visual channel, this paper only studied the cognitive load under the visual The experimental task was carried out in the Key Laboratory of Modern Manufacturing Technology of the Ministry of Education of Guizhou University, China, to keep the environment quiet and the light stable, eliminating all interference experimental factors.The study included a task with four layers of cognitive load, from a single channel to three channels.Specifically, the four tasks were as follows:

•
Visual channel: The sound equipment and handle of the emergency rescue system of the VR tunnel were switched off, and the tester obtained the rescue mission information only through the visual channel to complete the rescue mission.

•
Visual-auditory: The handle of the VR tunnel emergency rescue system was turned off, and the tester obtained rescue mission information through visual and auditory functions to complete the rescue mission.

•
Visual-tactile: The sound equipment of the VR tunnel emergency rescue system was turned off.
The tester obtained rescue mission information through visual and tactile sensation and completes the rescue mission.

•
Visual-auditor-tactile: The tester obtained the rescue information through visual reading; the auditory system acquires the tunnel rescue information, such as the tunnel wind sound, the water drops sounds, etc., and obtains the rescue information; the handle was touched to obtain the selected rescue information to complete the rescue task.
For each tester, random numbering was performed, and each tester had a preparation time of 1 min.The tester's task schedule is shown in Table 1.The experimenter completed the tunnel emergency rescue task through virtual reality equipment, and acquired the eye movement data in the process of completing the task by using the strap-back eye tracker of Xintuo Inki Technology Company.For example, the number of fixations, mean fixation duration, average pan length, number of fixation points at the first mouse click, and number of back-views were obtained.Subjective measurement and self-assessment is widely used as a measure of cognitive load [9,[36][37][38], which can detect small changes in cognitive load with a relatively good sensitivity [39].Therefore, at the end of the experiment, in order to verify the usability of the cognitive load evaluation model based on the probabilistic neural

Cognitive Channel Cognitive Load
Single As the number of cognitive channels changed, so does the eye movement index data, as shown in Table 5.In order to avoid repeated experiments and to remember the influence of the VR tunnel emergency rescue system environment and task on the cognitive load supervisor score, each participant could only complete one kind of modal cognitive experiment, such as the one-way to visual cognitive experiment, which was arranged as shown in Table 1.

Experimental Results
The cognitive load of the emergency rescue system in the VR tunnel in different cognitive channel environments was objectively evaluated.The results are shown in Table 6.

Dual Channel k=2
Three Channel k=3 Table 7 shows the data of eye movement indices during the emergency rescue of the VR tunnel under different cognitive channels, which have been normalized.

Correlation Analysis of Eye Movement Parameters and Cognitive Load of Users
Users' cognitive load obtained from a single type of eye movement data was limited and one-sided, which cannot accurately reflect the needs of users' interests.Therefore, it is necessary to integrate the data and establish a model of users' cognitive load based on an eye movement experiment.Additionally, it is necessary to analyze the correlation between eye movement data and the cognitive load.
In this paper, the Pearson correlation test was used to test the relationship between eye movement parameters and cognitive load, so as to improve the theoretical premise of the cognitive load evaluation.The results of the correlation analysis were obtained and can be viewed in Table 8.As can be seen from Table 8, the characteristic parameters of each eye movement index were significantly correlated with the cognitive load of users to varying degrees, and the high correlation between the eye movement index and the cognitive load is demonstrated once again.Average saccade length was more highly correlated with cognitive load than other parameters.

Model Output Analysis
Comparative analysis of the cognitive load evaluated by the probabilistic neural network model and actual cognitive load is shown in Figure 4, and the fitting degree is high.From Figure 4, it can be seen that the cognitive load evaluation model is close to the actual result, which indicates that the evaluation effect of this model is better.

Model Output Analysis
Comparative analysis of the cognitive load evaluated by the probabilistic neural network model and actual cognitive load is shown in Figure 4, and the fitting degree is high.From Figure 4, it can be seen that the cognitive load evaluation model is close to the actual result, which indicates that the evaluation effect of this model is better.At the same time, in order to understand the accuracy of the model used, the maximum absolute error and relative mean square error were used to evaluate the model, and the evaluation results are shown in Table 9.At the same time, in order to understand the accuracy of the model used, the maximum absolute error and relative mean square error were used to evaluate the model, and the evaluation results are shown in Table 9.In general, the mean absolute error was 10.7575% and the mean relative mean square error was 12.7675%.At the same time, it can be seen from the cognitive load evaluation results of each cognitive channel that the maximum absolute error was 16.01%, the minimum absolute error was 6.52%, the maximum relative mean square error was 23.21%, and the minimum relative mean square error was 6.64%.This shows that the cognitive load evaluation model based on the probabilistic neural network had a high precision, and the cognitive load model proposed in this paper had a good reliability and can accurately evaluate the cognitive load value of users under different cognitive channels, so as to effectively improve the design rate of the virtual reality interaction system and the user experience.

Conclusions
In this paper, the eye movement behavior of the experimenters in a virtual reality interactive environment was studied, and the cognitive load was calculated using the eye movement index such that the cognitive load could be quantified.Eye movement data were recorded using an eye movement instrument, and the subjective cognitive load of the current interactive system was investigated using a questionnaire.The conclusions are as follows.
Based on the experimenter's eye movement experiment, the number of fixation points, the average fixation duration, the average saccade length, the number of fixation points clicked during the first time, the number of backward-looking views, and other eye movement data were extracted, the user's cognitive load quantification model in the virtual reality interactive system was constructed by combining the probabilistic neural network.
From the results of the study, it can be seen that there was a significant correlation between each eye movement characteristic parameter and the cognitive load, which indicates that the eye movement index can directly reflect the cognitive load under the interaction of users, thus providing a basis for the study of cognitive load quantification.
The results show that the absolute error of the user cognitive load based on the probabilistic neural network and the subjective cognitive load value of the tester was 6.52%-16.01%,and the relative mean square error is 6.64%-23.21%,indicating that the method has a high precision.

Figure 1 .
Figure 1.Multi-modal interactive information integration model in a virtual reality system.

Figure 2 .
Figure 2. Eye movement assessment model of cognitive load in a virtual reality system.

Figure 1 .
Figure 1.Multi-modal interactive information integration model in a virtual reality system.

Information 2019, 10 , x 3 of 14 Figure 1 .
Figure 1.Multi-modal interactive information integration model in a virtual reality system.

Figure 2 .
Figure 2. Eye movement assessment model of cognitive load in a virtual reality system.

Figure 2 .
Figure 2. Eye movement assessment model of cognitive load in a virtual reality system.
eye movement index matrix is B = b ij y×s .Each column of the matrix represents eye movement indicator data, and each row represents a test value.As the units of each indicator data are different, it is difficult to directly compare the data, so it is necessary to normalize the data of each column, perform linear transformation of the original data, and map the result value to [0 − 1].If the cognitive load value increases with the increase of each set of indicator data, the transfer function is as follows:

Table 8 .
Correlation between each eye movement characteristic parameter and cognitive load.

Figure 4 .
Figure 4.The cognitive load evaluated by the probabilistic neural network model is compared with the actual cognitive load.

Figure 4 .
Figure 4.The cognitive load evaluated by the probabilistic neural network model is compared with the actual cognitive load.

Table 7 .
Normalized eye movement index data.

Table 9 .
Maximum absolute error and relative mean square error.

Table 9 .
Maximum absolute error and relative mean square error.