A Self-Adaptive Dynamic Recognition Model for Fatigue Driving Based on Multi-Source Information and Two Levels of Fusion

To improve the effectiveness and robustness of fatigue driving recognition, a self-adaptive dynamic recognition model is proposed that incorporates information from multiple sources and involves two sequential levels of fusion, constructed at the feature level and the decision level. Compared with existing models, the proposed model introduces a dynamic basic probability assignment (BPA) to the decision-level fusion such that the weight of each feature source can change dynamically with the real-time fatigue feature measurements. Further, the proposed model can combine the fatigue state at the previous time step in the decision-level fusion to improve the robustness of the fatigue driving recognition. An improved correction strategy of the BPA is also proposed to accommodate the decision conflict caused by external disturbances. Results from field experiments demonstrate that the effectiveness and robustness of the proposed model are better than those of models based on a single fatigue feature and/or single-source information fusion, especially when the most effective fatigue features are used in the proposed model.

of the fatigue driving recognition under a complex travel environment, models based on multi-source information have been developed. Boyraz et al. [19] proposed a recognition model using facial and vehicle behavior features. Yang et al. [20] proposed a Dempster-Shafer evidence theory (D-SET) based driver fatigue recognition model by combining contextual features, physiological features, and facial visual features. However, because these models ignore the temporal correlation of driver's fatigue state, their robustness reduces when some fatigue feature measurements cannot be obtained due to unexpected changes of external travel environment.
Other models attempt to capture the temporal correlation by applying basic probability assignment (BPA) in the information fusion to accommodate the external environment changes. Cheng et al. [21] proposed a two-level fusion framework that includes feature-level fusion and decision-level fusion for fatigue driving recognition. However, the accuracy of this fatigue driving recognition model is not reliable because the framework cannot dynamically determine the BPA for each expected fatigue state when multiple fatigue features are fused. Lee et al. [22] proposed a fatigue driving recognition model based on dynamic Bayesian network. While the model considers the recognition outcome of the previous time step, the probability weight of each fatigue feature is predetermined according to experts' subjective experience. The predetermined weights may deteriorate the accuracy of fatigue driving recognition if the experts have not factored the travel environment variation. Yang et al. [23] proposed a dynamic Bayesian network model in which the Gaussian distribution function is used to determine probability weights. A first-order hidden Markov model was used to compute the dynamics of the Bayesian network at different time steps. In their model, the probability weights are treated as constants after the assignment. As the probability weight of each fatigue feature in all these models cannot be changed dynamically to adapt to the environmental variation, the static weights may deteriorate the accuracy and robustness of the recognition model.
The literature review heretofore illustrates that the challenges in improving the accuracy and robustness of fatigue driving recognition model arise from disturbances such as sudden lighting changes, missing sensor signals, and incorrect feature measurements. This study seeks to improve the accuracy and robustness of fatigue driving recognition by developing a self-adaptive dynamic recognition model that incorporates the most effective fatigue features. In contrast to existing fatigue driving recognition models, the proposed model introduces a dynamic BPA to the decision-level fusion such that the weight of each feature source can adapt to real-time fatigue feature measurements. Additionally, the proposed model combines the most effective fatigue features in the feature-level fusion and the fatigue state at the previous time step in the decision-level fusion to improve the robustness of fatigue driving recognition. An improved correction strategy of the original BPA is introduced to accommodate the decision conflict caused by external disturbances.
The study contributes to the research on fatigue driving recognition in four aspects. First, the study is the first to determine the dynamic assignment of BPA by carrying out feature-level fusion based on Takagi-Sugeno fuzzy neural network (T-SFNN) [24]. This enables the improvement of robust fatigue driving recognition. Second, in addition to facial expression and vehicle behavior, the fatigue state at the previous time step is also regarded as an evidence source and incorporated into the model, which further enhances the robustness of recognition model through the consideration of the fatigue state correlation in temporal space. Third, a correction strategy for the original BPA is proposed in the decision-level fusion based on D-SET, which resolves the evidence conflict caused by multiple pieces of evidence with different attributes. Fourth, the fatigue driving recognition model is validated against data from field experiments instead of laboratory simulation, which demonstrates the practical applicability of the proposed model.
The remainder of this paper is organized as follows. Section 2 summarizes the fatigue feature measurements that are used to estimate driver fatigue state. Section 3 proposes a self-adaptive dynamic recognition model, including feature-level fusion based on T-SFNN and decision-level fusion based on D-SET. Section 4 demonstrates the effectiveness and robustness of the proposed model using the data collected from field experiments. The final section provides concluding comments.

Preliminaries
In this study, the fatigue driving state is estimated based on two categories of measurements: driver facial expression and vehicle operation condition. The two categories of measurements are explained in detail in this section.

Facial Feature Based Measurements
Blinking frequency (BF), eye-closed duration (ECD), mean of eye-opened level (MEOL) and yawning frequency (YF) are considered as facial fatigue features in this paper to indicate fatigue state of the driver.
The BF is measured as: where Nb is the number of the images captured in a one-minute interval, and nb is the number of the images where eyes are identified as closed state [25]. The ECD is measured as: /

Vehicle Behavior Feature Based Measurements
In this study, vehicle behavior features that are used to infer the driver's fatigue state include percentage of non-steering (PNS), standard deviation of steering-angle (SDSA), frequency of abnormal lane deviation (FALD) and standard deviation of vehicle speed (SDVS).  [16,27]: ( ) The SDVS is defined as: where νs represents the standard deviation of vehicle speeds, represents the average speed during a ten-second interval, Nν is the number of sample points in a ten-second interval and νν,i is the speed value gained by global positioning system (GPS) at time i. The measurements defined in this section will be used to determine fatigue features of the self-adaptive dynamic recognition model, which will be incorporated into the proposed recognition model for carrying out real-time fatigue driving recognition as input parameters.

Self-Adaptive Dynamic Fatigue Recognition Model
Because the multiple fatigue features determined in Section 2 come from two different fatigue feature sources, they can comprehensively reflect driver's fatigue state. In the following recognition model, we will incorporate all the measured fatigue features to enhance the reliability and robustness of fatigue driving recognition. However, if multiple fatigue features from different information sources are incorporated into a single-level recognition model, it can result in a complex model structure and weak system stability. To enhance the model performance, we propose a self-adaptive dynamic recognition model with two levels of fusion, which includes the feature-level fusion based on T-SFNN and decision-level fusion based on D-SET. In the feature-level fusion, T-SFNN is used to fuse multiple fatigue features obtained from facial expression and vehicle operation behavior to provide a dynamic BPA for the decision-level fusion. In the decision-level fusion, D-SET is used to fuse three pieces of evidence derived from three different information sources to improve the robustness of fatigue driving recognition.

General Recognition Framework
The model structure of the proposed self-adaptive dynamic recognition model is summarized in Figure 1. The key components and recognition procedures of the fatigue driving recognition model are as follows. In the following sections, we will discuss the feature-level fusion and decision-level fusion in detail.

Feature-Level Fusion
The T-SFNN combines the advantages of fuzzy logic (in processing vague and uncertain information) and neural networks (in providing good learning capabilities) [24]. Compared with the conventional approaches, e.g., fuzzy logic, it has been shown that TSFNN can achieve better performance in mathematical function approximation in modeling highly nonlinear systems [24]. Therefore, the feature-level fusion based on T-SFNN model can generate more accurate BPA for decision-level fusion. In this study, T-SFNN-1 and T-SFNN-2 models are used to generate real-time BPAs by fusing facial and vehicle behavior features, respectively. For simplifying the model structure, in each T-SFNN model, the subtractive clustering algorithm (SCA) is used to obtain the optimal T-SFNN structure. The improved particle swarm optimization (IPSO) algorithm is also employed to train the T-SFNN for acquiring accurate network parameters. Therefore, based on the simplified structure and optimized parameters, the two T-SFNN models can enable more accurate and reliable BPAs.

Structure of T-SFNN
Traditional T-SFNN consists of antecedent network and consequent network with complex network structure and numerous network parameters to be determined [24]. To simplify the network structure and improve computational efficiency, traditional T-SFNN is improved by reorganizing the network structure. As shown in Figure 2, the improved T-SFNN structure is composed of five layers and the function of each layer is as follows: The first layer is the input layer, where each node represents an input variable. The second layer is the fuzzy layer, where each node represents one linguistic value of each input variable. Each node expresses a fuzzy subset j i Q described by Gaussian membership function: linguistic values of each input variable, n is the number of nodes in the input layer, and cij and σij are defined as the center and width of the Gaussian membership function, respectively. The third layer is the rule layer which is used to calculate the firing strength αj of every fuzzy rule. Each node in the layer represents one fuzzy rule, The fourth layer is the normalized layer which is used to calculate the normalized firing strength j α of the corresponding rule: The fifth layer is the output layer which is used to provide the reference result of the whole system.
Each node in this layer represents one output variable, where r is the number of output variables.

Learning Algorithm
For the T-SFNN shown in Figure 2, when the number of input variables and fuzzy sets for each input becomes more than four or five, it will result in the problem of combinatorial explosion of rules, namely the so-called curse of dimensionality. In order to optimize the inner structure of the T-SFNN and obtain optimum network parameters, we introduce a structure learning algorithm based on SCA and parameter learning algorithm based on IPSO.
For the structure learning, the SCA is used to optimize the network structure according the cluster amount extracted from the given training samples [28]. By clustering, let the number of linguistic values of each input node and the number of fuzzy rules be equal to the number of the extracted clusters, which will significantly reduce the number of parameters of the TSFNN. The algorithm used in this study can be succinctly described as follows. Suppose that is the jth cluster center obtained by SCA, where cR is the number of clusters determined by the SCA. Assuming that C1 is the first cluster center obtained, and Cs is the nearest cluster center to C1, i.e., the Euclidean distance between C1 and CS is the shortest, where, 1 , the initial value ij c′ and ij ′ σ of parameters cij and σij are determined. For the parameter learning, the back propagation (BP) algorithm [29] and genetic algorithm (GA) [30] have been proven to have superior performance for specific problems. However, because of gradient descent, the BP method has been criticized for its shortcomings of becoming stuck in local minima and sensitivity to the initial values. The main drawbacks of GA are its huge computation time and slow convergence near the optimum. When compared to BP and GA, the IPSO is simpler in operation and easier to understand owing to a smaller number of free tunable parameters. Therefore, we introduce the IPSO algorithm to determine network parameters and network weights of T-SFNN. First, determine the initial value of network parameters. The network parameters are initialized as ij c′ and ij σ ′ according to the SCA, and network weight wkj is initialized as a random value kj w′ in the range of [0,1] . Second, the IPSO algorithm is used to obtain the optimal solutions of network parameters and network weight. In the IPSO, according to the fitness function shown in Equation (1), the optimal solutions of ˆi j c , ˆi j σ and ˆk j w are obtained [28].
where N is the number of training samples, r is the number of the nodes in the output layer, and yk,j and ,k j y are the actual and desired outputs, respectively.

Decision-Level Fusion
Multi-source information fusion based on Bayesian networks has been applied to the field of fault diagnosis and achieved good diagnosis outcomes [31]. Compared to other statistical inference methods, D-SET is closer to human perception and reasoning process, and can fuse information collected from different sources to infer results with some degree of certainty. Unlike the Bayesian reasoning model that depends on the prior probability, D-SET is more suitable for practical applications [32]. Therefore, based on the D-SET, we propose an improved decision-level fusion method to combine three pieces of evidence from three different information sources. The proposed decision-level fusion method includes three steps: (i) dynamic BPA calculation; (ii) combination of evidence; and (iii) fatigue state decision.

Dynamic BPA Calculation
The key role of D-SET is to model the knowledge of the problem by initializing the BPA based on the evidence provided by different sensors. Assume m(Ai) is the BPA of the th i hypothesis Ai in the frame of discernment Θ of D-SET, where Θ is a finite non-empty set of mutually exclusive alternatives containing every possible hypothesis Ai [32]. Notation m is a mass function, m(·): , which satisfies: [33]. In this paper, Ai represents the evaluated fatigue state according to a certain fatigue assessment method. Generally, the BPA is determined by experts based on their experience. However, such a subjective and static BPA assignment will reduce robustness of the proposed model in practice. To overcome the difficulty, the BPA m(Ai) for each evidence source is dynamically assigned by the T-SFNN model according to its real-time output results in this study. The m(Ai) can be calculated by: where yi is the output of the th i node component in output layer of the T-SFNN.

The Improved Evidence Combination
Denote Θ ={ 1 A , 2 A , , r A } as fatigue state set, e ={ 1 e , 2 e , , t e } as evidence set, and m1(·), m2(·),  , mt(·) as the BPA mass functions over Θ. The Dempster combination rule for any two pieces of evidence in the evidence set e is [34,35]: , which denotes the conflict degree between two pieces of evidence.
In Equation (3), K = 1implies that the two pieces of evidence are completely conflicting. Under this situation, the aforementioned combination rule becomes invalid. If 1 K → , then two pieces of evidence have high conflict and an illogical result may be produced by this rule [36]. Therefore, we should identify whether a conflict happens before conducting evidence combination based on the conflict degree K. The improved evidence combination rule for any two pieces of evidence is given: if 0 T K k < < , then the original BPA is available in the following evidence combination, where kT is a predetermined threshold. Otherwise, if 1 T k K ≤ < , then the original BPA needs to be modified by introducing a belief factor. The modified BPA is updated as: In Equation (4), the belief factor ηi can be obtained through the following steps: Step 1: Calculate the Euclidean distance di,j between any two pieces of evidence ei and ej: Step 2: Define confidence level , Step 3: The average confidence level ci can be defined as: Step 4: The belief factor ηi is determined by:

Fatigue State Decision
After conducting the evidence combination, we can determine the driver's fatigue state based on the following decision rule: where m(AF) is the largest probability value, ( ) max{ ( ), Θ} If AF satisfies the decision rule Equation (6), then it is regarded as the driver's fatigue state; otherwise, the fatigue state recognized in the previous time step is outputted as the current fatigue state.

Experiment Results and Analysis
This section presents examples based on field experiments to demonstrate the effectiveness and robustness of the proposed fatigue recognition model by applying the field experiment data to the proposed model and comparing its performance with those of other models.

Experiment Design
The experiments were carried out on the Nanjing-Shanghai expressway (China), which is highlighted in blue in Figure 3. Three men and two women with ages ranging from 25 to 32 and more than three years of driving experience participated in the experiments. Alcohol, tea, coffee, drugs or other drinks that can cause excitement to the nervous system were prohibited for 24 h prior to the experiments. The experiments were performed after informed consent on the procedures of the experiments was received from all participants. The experiments were carried out from 12:00 p.m. to 3:00 p.m. on 25 November 2014 for all participants, because previous studies show that drivers can become easily fatigued during this period of the day [37]. To avoid traffic accidents, the experiments were conducted on road sections with few vehicles. In addition, an experienced driver was asked to sit in the front passenger seat to warn the participant or execute necessary emergency maneuvering.

Fatigue State Assessment
The effective classification of a driver's fatigue state during driving is a difficult problem. The existing studies focus on subjective assessments on fatigue driving by observing some phenomena, such as a driver's facial expression, operational behavior and self-evaluation of fatigue, which may result in an inaccurate classification. In view of objectivity of the EEG detection, we propose a comprehensive assessment method based on objective detection and subjective assessment to classify fatigue states, which will improve the accuracy of fatigue state classification. The main steps of the comprehensive assessment method are as follows: (i) Observer Assessment: Fatigue states of participants are evaluated by three observers according to the video of facial expression and driving behavior captured by three cameras. Of the three cameras, one is placed towards the driver's face, another towards the steering-wheel, brake pedal, and gearshift, and the third one towards the front lane. Every video is evaluated by the three observers according to the fatigue state characteristics described in Table 1. If the fatigue state of the participant is evaluated as "Non-Fatigue (NF)", then let si,j = 0, where si,j indicates the score of the jth video evaluated by the th i observer; if the fatigue state is determined as "Moderate Fatigue (MF)" or "Severe Fatigue (SF)", then let si,j = 1 or si,j = 2, respectively. To reduce subjectivity of assessment, the average of the scores given by three observers is regarded as the final score, i.e., where, Pα, Pθ and Pβ are the power spectra of the three wave bands of α, θ and β, respectively, and the frequency ranges of α, θ and β bands are (4-8 Hz), (8)(9)(10)(11)(12)(13), and (13-22 Hz), respectively. The fatigue state is categorized into three levels according to the value of rα,θ,β defined in Table 2.
(iii) Self-Assessment: Let the participant make a self-assessment of fatigue state according to his/her current physical, physiological and psychological situations. Based on the scores gained and the 7-point Stanford Sleepiness Scale (SSS) table [39], fatigue is then rated into one of three states: "NF" (1-2 points), "MF" (3-5 points), and "SF" (6-7 points).  3 4 r α θ β ≤ < SF , , 4 r α θ β ≥ (iv) Comprehensive Assessment: The participant's fatigue state is determined according to the results from steps (i)-(iii). If the fatigue state results from all three methods are consistent, then the assessment result obtained is considered valid and correct, which will be regarded as the actual fatigue state of the participant. Otherwise, it is removed from the ground truth set.
To ensure the reliability of the assessment, the assessments based on observer and EEG detection must be implemented simultaneously, and the self-assessments based on SSS table need to be carried out within one minute after the other two assessments are accomplished.
The proposed assessment method will be used to select the most effective fatigue features and provide ground truth data for following model calibration and verification.

Data Collection
According to the detection methods described in Section 2, eight fatigue features were measured. In total 1200 data points were collected, each of which includes the eight fatigue features. These data were separated into two sets: (i) the training set that included 800 data points for model calibration; and (ii) the remaining 400 data points for model verification. Further, based on the comprehensive assessment method proposed in Section 4.2, the measurement results of every fatigue feature were divided into three groups according to their corresponding fatigue states, "NF", "MF" and "SF". We select 150 data points randomly from the 1200 data points illustrated in Figure 4. For each fatigue feature, 50 data points each belong to the "NF", "MF" and "SF" groups.   As seen from the Figure 4, the fatigue states cannot be identified easily according to the measurement values of fatigue features. For example, theoretically, the data in the "NF" state should have larger values than the data in the "MF" state, and the data in the "MF" state should have larger values than the data in the "SF" state in MEOL measurement shown in Figure 4c. However, some data points in the "NF" state have smaller values than the data in the "MF" state. Therefore, in order to recognize the driver's fatigue state more effectively, the effectiveness of these measured fatigue features needs to be analyzed further.

Fatigue Feature Identification
As previously mentioned, not all fatigue features measured in the previous section can be used to reliably reflect fatigue state of driver. In addition, too many fatigue features are likely to cause data redundancy and result in heavy computational burden, which may preclude real-time recognition. Therefore, the fatigue features presented in the previous section need to be analyzed further to eliminate irrelevant fatigue features.
To select the most effective fatigue features, the correlation between fatigue features and fatigue driving need to be identified. Pearson test is regarded as an effective method to carry out the correlation analysis. The associated procedure is as follows: (i) The Kolmogorov-Smirnov test [40]: It is used to estimate whether every fatigue feature follows a normal distribution. For the features not normally distributed, they will be normalized through logarithmic transformation. (ii) Pearson test [40]: It is used to verify the correlation between a fatigue feature and fatigue. (iii) Feature selection: The most effective fatigue features are selected according to the correlation calculated by Pearson test. If the statistic value of a fatigue feature is smaller than the quantile value, it is considered uncorrelated to fatigue driving and removed from the candidate set of fatigue features. Table 3 shows the verification results based on the Kolmogorov-Smirnov test. It shows that the statistics of BF, ECD, MEOL, YF, PNS, SDSA, FALD and SDVS are smaller than the quantile value, implying that they are all normally distributed. The correlation coefficient between each fatigue feature and fatigue state is computed by using the Pearson test and the associated statistics are summarized in Table 4. Table 4 illustrates that the correlation coefficients of MEOL and SDSA are negative and the other six fatigue features are positively correlated to fatigue. Further, the absolute value of the correlation coefficient of MEOL is very small and its statistic value is less than the corresponding statistic quantile value. It indicates that MEOL is not significantly correlated with fatigue. Therefore, the seven fatigue features that have significant correlation with fatigue are selected as the most effective fatigue features, and the fatigue feature MEOL is excluded.

Feature-Level Fusion Results
There are four main steps in the feature-level fusion. First, according to the number of the effective fatigue features and the fatigue state of sample data, a 3-input and 3-output neural network of T-SFNN-1 is determined, where input variables xi,1 (i = 1, 2, 3) are used to represent the BF, ECD and YF measurements, respectively. A 4-input and 3-output neural network of T-SFNN-2 is determined, where input variables xi,2 (i = 1, 2, 3, 4) are used to represent the PNS, SDSA, FALD and SDVS measurements, respectively. Output variables yj ( j = 1, 2, 3) represent the probabilities of the three fatigue states, i.e., the "NF", "MF", and "SF", respectively. The probabilities are assigned according to the fatigue state of the sample data in training the T-SFNN. If the data point in the training set is evaluated as a certain fatigue state by the proposed comprehensive assessment method, then the output variable related to the evaluated fatigue state is assigned a larger probability value. The remaining probability value is assigned equally to the two output variables related to the other two fatigue states. For example, if the fatigue state of the th i sample data is evaluated as "NF", then we can let y1(i) = 0.8, y2(i) = 0.1, y3(i) = 0.1; if the fatigue state is "MF", then let y1(i) = 0.1, y2(i) = 0.8, y3(i) = 0.1; if the fatigue state is "SF", then let y1(i) = 0.1, y2(i) = 0.1, y3(i) = 0.8.
Second, based on the SCA and sample data, determine the structures of T-SFNN-1 and T-SFNN-2. 480 sample data points are selected from the training set and are used to carry out the SCA. Based on the SCA, these data points for T-SFNN-1 and T-SFNN-2 are divided into 3 clusters. Therefore, the number of fuzzy rules is determined as 3 and the number of the linguistic values for every input variable is also 3 in both T-SFNN-1 and T-SFNN-2. The network structures of the T-SFNN-1 and the T-SFNN-2 without and with the SCA are shown in Table 5. Table 5 illustrates that the structures of T-SFNN-1 and T-SFNN-2 have been improved when the SCA is adopted. The total number of parameters to be determined in the training stage decreases significantly. Table 5. Comparison of network structure of T-SFNN without and with SCA. Third, T-SFNN-1 and T-SFNN-2 are trained to obtain the optimal network parameters using the IPSO algorithm based on 360 training sample data points, including 120 "NF", 120 "MF" and 120 "SF" samples, respectively. The convergence curves measured by the mean square error (MSE) are shown in Figure 5. As illustrated by Figure 5, the MSE values of T-SFFN-1 and T-SFNN-2 decline to 10 −4 after 1642 iterations based on the IPSO algorithm. Therefore, the network parameters can be efficiently determined using the IPSO algorithm.

Decision-Level Fusion Results
The normalized results of 1 Further, the selected 200 data points were used to verify the robustness of the proposed model under disturbance. Table 7 summarizes the recognition results of the first evidence fusion based on the results shown in Table 6, where F s′ represents the driver's fatigue state deduced from {ṁ t (A1), ṁ t (A2), ṁ t (A3)} according to the proposed decision rule in Equation (6), SEEG represents the fatigue state evaluated by the proposed comprehensive assessment method, and K′ represents the conflict degree between two pieces of evidence in the first evidence fusion. Table 8 shows the recognition results of decision-level fusion of the 6 examples, where K represents the conflict degree calculated in the decision-level fusion. As shown by Table 7, the first evidence fusion results differentiate the fatigue states more clearly than the results provided by the feature-level fusion shown in Table 6. For example, for the 2nd sample data point in Table 6, the output value 1,2 t y (0.763) of T-SFNN-1 is larger than the 1,1 t y (0.203) and 1,3 t y (0.052); that the fatigue state belongs to "MF" can be recognized easily. By contrast, the difference between 2,1 t y (0.402) and 2,2 t y (0.432) is small. Hence, it is difficult to conclude the driver's fatigue state based on T-SFNN-2. Nevertheless, after the first evidence fusion is conducted as shown in Table 7, the probability ṁ t (A2) (0.784) of "MF" is much larger than the other two probabilities ṁ t (A1) (0.193) and ṁ t (A3) (0.026). Hence, the driver's fatigue state can be determined based on the proposed decision rule in Equation (6).
For the 4th sample data point, 1,2 t y has the maximum 0.488 in the output of T-SFNN-1, and the driver's fatigue state should be recognized as "MF". However, as 2,1 t y in T-SFNN-2 has the maximum 0.721, the driver's fatigue state will be recognized as "NF", which conflicts with the result of T-SFNN-1. By contrast, the first evidence fusion result in Table 7 indicates that the driver's fatigue state is "NF" because ṁ t (A1) (0.708) is larger than ṁ t (A2) and ṁ t (A3). The proposed comprehensive assessment shows that driver's actual fatigue state is "NF", which is consistent with the first evidence fusion result. Using video data analysis, we find that the reason why T-SFNN-1 was unable to correctly infer the fatigue state is that the fatigue feature ECD was incorrectly measured due to the driver's sudden nodding.
For the 5th sample data point, the recognition results of T-SFNN-1 and T-SFNN-2 are also different. One is "SF" while the other is "MF". By checking the video records, we found that the fatigue feature FADL was incorrectly measured due to the blurred lane and dim lighting, which resulted in the failure of T-SFNN-2 recognition. By contrast, the first evidence fusion result can correctly recognize the driver's fatigue state. Table 8 summarizes the final results of the proposed model after the decision-level fusion. The performance of the two levels of fusion model is enhanced because the distinction between m t (A1), m t (A2) and m t (A3) is enhanced. For example, for the 1st sample data point, the difference of the maximum ṁ t (A1) and the second maximum ṁ t (A2) is increased to 0.883 from 0.495 based on the first evidence fusion result. This enhancement makes the recognition of fatigue state more credible. For the 6th sample data point, the driver's fatigue state is recognized as "MF" according to the first evidence fusion result shown in Table 7. However, the fatigue state of the 6th sample data point is determined as "SF" according to the decision-level fusion result shown in Table 8. The comprehensive assessment proposed verifies that the actual fatigue state is "SF", which is consistent with the decision-level fusion result. By analyzing the video data, we found that the GPS device did not receive any signal of vehicle position and velocity because the vehicle entered a tunnel, which resulted in a recognition failure of T-SFNN-2. By contrast, the decision-level fusion was able to obtain correct fatigue recognition by considering the fusion result at the previous time step. Finally, to demonstrate the accuracy and robustness of the proposed model, it is compared with the models based on the single feature, and the single-source fusion from three perspectives: accuracy rate (AR), miss rate (MR), and false alarm rate (FAR). Here, AR = (N0,0 + N1,1 + N2,2)/N, FAR = (N0,1 + N0,2)/N, MR = (N1,0 + N2,0)/N, where N is the total number of the samples, and Ni,j is the number of sample data points recognized as having fatigue state j when the actual fatigue state is i. The performance of the proposed model is also determined based on two sets of feature measurements: (i) using all of the fatigue features; and (ii) using only the most effective fatigue features (excluding MEOL). The results are summarized in Table 9, and illustrate that the proposed model is better than the other models in terms of AR, MR and FAR. Further, while the number of the fatigue features is reduced through fatigue feature identification, the performance of the model is improved. It indicates that the proposed model can provide more accurate and robust results in real-world applications. Through these analyses, we conclude that the proposed self-adaptive dynamic recognition model with two levels of fusion is effective and robust, even when certain fatigue features become ineffective or some sensors fail because of complex travel environment. In addition, the use of the most effective features can further improve the performance of the proposed model.

Conclusions
To enhance the effectiveness and robustness of fatigue driving recognition, a self-adaptive dynamic recognition model based on multi-source information and two levels of fusion is proposed in this paper. The accuracy and robustness of fatigue driving recognition are improved through the feature-level fusion as well as the decision-level fusion. The feature-level fusion based on T-SFNN can provide an accurate dynamic probability assignment for the decision-level fusion, while the decision-level fusion based on D-SET is able to adaptively solve the decision conflict caused by external disturbances via combining three pieces of evidence from three different information sources. In addition, the proposed fatigue recognition model is calibrated and verified using a comprehensive assessment method of the fatigue state and the data collected from field experiments.
The experiment results demonstrate that the proposed model performs well in terms of accommodating the disturbances caused by complex environment changes. When the most effective fatigue features are selected and applied to the proposed model by conducting a correlation analysis, the performance of the model is further improved in terms of accuracy and reliability. In addition, compared to models based on the single fatigue feature and/or single-source fusion, the proposed fatigue recognition model provides more accurate and robust results in terms of the accuracy rate, missing rate, and false alarm rate. Therefore, the proposed fatigue recognition model can perform better in real-world applications to improve travel safety.