Developing a Deep Neural Network for Driver Fatigue Detection Using EEG Signals Based on Compressed Sensing

Abstract

In recent years, driver fatigue has become one of the main causes of road accidents. As a result, fatigue detection systems have been developed to warn drivers, and, among the available methods, EEG signal analysis is recognized as the most reliable method for detecting driver fatigue. This study presents an automated system for a two-stage classification of driver fatigue, using a combination of compressed sensing (CS) theory and deep neural networks (DNNs), that is based on EEG signals. First, CS theory is used to compress the recorded EEG data in order to reduce the computational load. Then, the compressed EEG data is fed into the proposed deep convolutional neural network for automatic feature extraction/selection and classification purposes. The proposed network architecture includes seven convolutional layers together with three long short-term memory (LSTM) layers. For compression rates of 40, 50, 60, 70, 80, and 90, the simulation results for a single-channel recording show accuracies of 95, 94.8, 94.6, 94.4, 94.4, and 92%, respectively. Furthermore, by comparing the results to previous methods, the accuracy of the proposed method for the two-stage classification of driver fatigue has been improved and can be used to effectively detect driver fatigue.

1. Introduction

With the advancement of industrial technology in recent years, car production has increased dramatically, resulting in an increase in traffic accidents. Every year, 1.25 million people die in road accidents, according to the World Health Organization (WHO) [1]. Driver fatigue can be considered the main cause of road fatalities among the factors affecting car accidents. According to the National Highway Traffic Safety Administration (NHTSA), 100,000 driver fatigue accidents caused 1550 deaths, 71,000 injuries, and USD 12.5 billion in monetary losses, annually, in the United States [2]. Therefore, it is important to develop a method that can detect the levels of mental fatigue accurately and automatically in order to prevent catastrophic driving events [3].

Fatigue may be caused by a lack of sleep, prolonged driving, driving overnight, driving on a monotonous route, and so on. Fatigue, as a general term, involves drowsiness. Drowsiness is defined as the need for sleep, while fatigue requires rest (not necessarily sleep) [3,4,5]. Yawning, impatience, daydreaming, heavy eyes, etc., are the initial signs of fatigue [5,6,7]. Driver fatigue slows down the reaction time (prolongations), it reduces the driver’s alertness, and it affects the driver’s decisions. Fatigue on the road can be suppressed by listening to music, drinking coffee or energy drinks, and so on [7,8,9]. However, it is necessary to design an automatic system to detect the driver’s mental fatigue with a high reliability that can warn the driver before potential accidents. In recent years, many studies have been conducted on the automatic detection of driver fatigue. Still, for various reasons, including inaccessibility and poor performance, the proposed systems have not yet entered the real world.

Automated detection-of-driver-fatigue systems fall under three general categories: The first category uses vehicle parameters, such as the standard deviation of the lane position and the steering wheel movement, which are affected by external factors, such as road markings, as well as the geometric features of the road [10,11,12]. The second category is based on driver behavior measurements, which include facial expression and eye tracking. However, these systems rely on lighting conditions [3,9]. The third category is based on physiological signals, which provide an objective and accurate method for measuring driver drowsiness and mental fatigue. These measurements are based on the fact that physiological signals begin to change in the early stages of fatigue, allowing the driver-fatigue-measurement systems extra time to alert the fatigued driver. The physiological signals that are used for the automatic detection of driver fatigue are measured using electrooculography (EOG), electroencephalogram (EEG), and etc. Among the physiological signals used to automatically detect driver fatigue, EEG signal analysis is shown to be the most promising method in recent studies because of its noninvasive, accessible, and accurate nature [13]. In the following section, the most recent studies for the automatic detection of driver fatigue on the basis of EEG signals are reviewed.

Correa et al. [14] extracted time-frequency features from the EEG signals to automatically detect driver fatigue. The accuracy of their classification, which is based on artificial neural networks (ANNs), is reported to be 83%. Xiong et al. [15] used approximate entropy (AE) and sample entropy (SE) to detect driver fatigue from the EEG signals. They performed their experiment on 50 subjects. These researchers used a support vector machine (SVM) to classify two stages of driver fatigue, and the classification accuracy of their proposed method is reported to be 90%. Chai et al. [16] used an autoregressive (AR) model to automatically detect driver fatigue from the EEG signal. The experiments were conducted on a total of 48 participants, on the basis of two classes of driver fatigue. The researchers also used independent component analysis (ICA) to reduce the feature vector’s dimensions in their research. The accuracy of their classification is reported as 88.2%, on the basis of the Bayesian classifier. Zhang et al. [17] identified two classes of driver fatigue using a combination of EEG, EMG, and EOG signals on 20 subjects. These researchers extracted several time-frequency features from the signals and achieved 90% accuracy, on the basis of the ANN classifier. Yin et al. [18] used fuzzy entropy (FE) to automatically detect driver fatigue on the basis of EEG signals. The researchers experimented with six participants to classify two stages of driver fatigue. The accuracy of their classification is reported to be 9%, on the basis of the ANN classifier. Ko et al. [19] used the fast Fourier transform (FFT) to extract the EEG signal’s frequency features to automatically detect driver fatigue. They used 50 subjects for the experiment, which was based on a virtual reality (VR) system. The classification accuracy, based on the linear regression (LR) model, is reported to be 90%. Wang et al. [20] used power spectral density (PSD) to extract features from the EEG signals in order to classify driver fatigue. The accuracy of their classification, based on the LR model, is reported to be 83%. Mou et al. [21] used FFT to extract frequency features to automatically detect driver fatigue on the basis of EEG signals. These researchers experimented on 20 subjects to classify two stages of driver fatigue. Their classification accuracy is reported to be about 85%. Nugraha et al. [22] extracted several time-frequency and statistical features from the EEG signal, including the mean, standard deviation, correlation, FFT, etc., to automatically detect driver fatigue. In their experiment, the Emotive EPOC+ system was used to identify two-stage driver fatigue in 30 subjects. The accuracy of their classification is reported to be 96%. Hu et al. [23] used AE, SE, and FE to detect driver fatigue from the EEG signals. They performed their experiment on eight subjects. Their classification accuracy is reported to be 92%, on the basis of the AdaBoost classifier. Min et al. [24] used the AE, SE, and FE features of the EEG signals to detect driver fatigue. The accuracy, sensitivity, and specificity of their classification were 98.3, 98.3, and 98.2%, respectively, on the basis of the ANN classifier. Cai et al. [25] used horizontal visibility graph (HVG) theory to detect driver fatigue on the basis of EEG signals. These researchers performed their experiments on 28 subjects to classify two states of driver fatigue, and they used the EEG Lab toolbox to eliminate the environmental and motion noise. The accuracy of their classification was reported to be approximately 98%. Luo et al. [26] used a combination of the adaptive scaling factor and entropy features to automatically detect driver fatigue. The researchers performed their experiments on 40 subjects using two channels of the EEG signals (Fp1 and Fp2) for a two-stage classification of driver fatigue. The EEG toolbox was also used to remove EOG noise. Moreover, the classification accuracy of their algorithm is reported to be about 98%. Gao et al. [27] used deep neural networks (DNNs) to detect driver fatigue on the basis of EEG signals. The researchers also performed their experiments on 10 subjects. Their deep network architecture consisted of 11 convolutional layers. The accuracy of the classification reported by these researchers is about 95%. Karuppusamy et al. [28] used a combination of EEG signals, facial expressions, and gyroscopic data to diagnose driver fatigue. The researchers used DNNs in their research. The final accuracy obtained, according to the model proposed by these researchers, is approximately 93%. Jiao et al. [29] used EEG and EOG signals to automatically detect driver fatigue. The researchers used continuous wavelet transform (CWT) to extract the frequency band analysis, and then selected the discriminatory features in their proposed model. They also used generative adversarial networks (GANs) to balance the samples of their class. The final accuracy reported by these researchers, according to the long short-term memory (LSTM) classifier, is approximately 98%. Liu et al. [30] used deep transfer learning networks to automatically detect driver fatigue on the basis of EEG signals. The random forest (RF) algorithm was also used to identify active channels. The highest accuracy reported by these researchers is approximately 73%.

A comprehensive review of previous studies for the automatic detection of driver fatigue indicates that, while several studies have been conducted in this regard, there are still some issues from various perspectives that should be considered: (1) The majority of these studies extract features manually, and few studies have used feature learning methods to extract features. Using manual methods necessitates complex processes, as well as specialized knowledge. Furthermore, manual feature extraction does not guarantee that the features chosen are optimal; (2) In most feature-learning-based driver-fatigue-detection methods, a large amount of raw time signals from multiple electrodes at high sampling rates are used directly as the input of the feature learning algorithms, imposing a significant burden on the acquisition hardware, data storage, and transmission bandwidth. As a result, it is critical to fundamentally alter the data processing mode of existing real-time monitoring systems by employing a novel data acquisition method and compression theory.

The current study aims to overcome the abovementioned challenges, especially the problem of a large amount of raw time signals. To the best of the authors’ knowledge, this research study presents, for the first time, a novel method for the automatic detection of driver fatigue using a combination of compressed sensing (CS) and DNNs. Recently, CS has attracted the attention of many researchers in this field of research, and has shown great success in various fields, such as magnetic resonance imaging (MRI) [31], radar imaging [32], and seismic imaging [33]. DNNs are also a group of machine learning methods that can learn features hierarchically, from lower levels to higher levels, by building a deep architecture. Deep learning has achieved state-of-the-art performance in several application domains, such as signal processing [34,35,36], network security [37], the Internet of Things (IoT) [38], and so on.

In the proposed method, a deep convolutional long short-term memory (DCLSTM) neural network is designed to learn the optimal features from compressed data. In this study, compressed EEG signal data is used to detect driver fatigue for the first time. Indeed, the most significant contribution of this study can be attributed to the reduction in the amount of data collected in order to obtain the optimal features, while retaining useful information in the compressed data. In addition, by using compressed data as input, the computational burden of the feature learning process is significantly reduced. The simulation results of the proposed method for a single-channel recording show an accuracy of 92% for a compression rate of 90, which is a significant accuracy. For more clarity, the following outlines the contributions made by this article:

a.

It provides a new and fully automated method for selecting and extracting discriminative features from the EEG signal to identify two stages of driver fatigue, with a high performance based on DNNs, with a proposed deep architecture, that does not require prior knowledge of, or expertise on, each case/subject;

b.

For the first time, the paper presents the application of CS theory in combination with DL to reduce EEG signal samples without the loss of essential signal information, related to automatic driver-fatigue detection, for use in real-time systems;

c.

The article illustrates the use of a minimum number of EEG signal channels to detect driver fatigue automatically, with the precondition of high classification accuracy and low detection errors;

d.

The selection of the parameters of the proposed method, and the effect of the key parameters on the deep network architecture, were thoroughly investigated in order to automatically detect driver fatigue. Furthermore, comparisons with traditional methods show the superiority of our proposed method. Because of the use of CS in the proposed algorithm, this method is suitable for extensive data processing and real-time processing, and it also provides a new idea for smart driver-fatigue detection;

e.

In this study, the environmental noise while driving was considered for the first time among the previous research related to driver-fatigue detection. The results show that the proposed network is robust to noise up to 1 dB.

The rest of the paper is organized as follows: In Section 2, the mathematical background of CS theory and DNNs is presented; in Section 3, the acquisition of the EEG data, the proposed method, the parameter selection, and the network architecture are examined. Section 4 discusses the simulation results and makes comparisons to previous research, and, finally, Section 5 consists of the conclusion.

2. Background

In this section, a brief mathematical introduction to CS theory and DNNs for the automatic detection of driver fatigue is described.

2.1. Compressed Sensing Theory

This section provides a brief description of CS theory [39]. Theoretically, CS adopts a quasi-optimal measurement scheme that gathers all the raw signal information and that realizes an efficient and dimensionally reduced representation of the signal. Using CS theory, a sparse signal can be reconstructed from far fewer samples than what the Shannon–Nyquist sampling theorem [40] requires. It has been shown that many nonsparse signals (such as EEG signals) can be converted to sparse signals through various signal transformations, such as discrete cosine transformation (DCT), and discrete wavelet transformation (DWT). In view of the above, the CS theory can also be applied to nonsparse signals, provided that the sparsity of the transformed signal is guaranteed. Considering $X\in R^N$ and $\Phi \in R^{M\times N}$ as the input signal and the measurement matrix, respectively, the compressed output, $Y\in R^M$, is obtained as:

$$Y=\Phi X$$

(1)

A measurement matrix that satisfies the restricted isometry property (RIP) will result in $M\ll N$, which means an output observation vector of much less dimensionality than that of the input signal. The RIP implies that, for any strictly sparse vector, the measurement matrix must satisfy:

$$1-\delta \le \frac{{\Vert \Phi X\Vert }_2^2}{{\Vert X\Vert }_2^2}\le 1+\delta$$

(2)

where δ is the RIP constant, which has a value between 0 and 1. The compression ratio (CR) is an indicator of the extent to which the signals are compressed, and is defined as:

$$CR=\frac{N-M}{N}\times 100\%$$

(3)

Thus, having $M\ll N$ will result in a high CR. Thus, various levels of compressed acquisitions are achieved by adjusting the size of the measurement matrix (provided that the RIP condition is met). Finally, an exact reconstruction of $X$ can be performed by various algorithms, such as L1 norm minimization [41], and orthogonal matching pursuit (OMP) [42].

2.2. Deep Convolutional Neural Networks

CNNs are considered powerful deep learning techniques to learn sophisticated phenomena. CNNs are very efficient in terms of learning features, and they are used in a variety of applications, such as computer vision. CNNs consist of three main layers: convolutional layers, pooling layers, and fully connected layers (FCs) [43,44]. In the convolutional layers, the CNN network uses various kernels to convolve the input signal, as well as intermediate feature maps. These layers reduce the number of parameters and cause immutability and stability with respect to displacement. The pooling layer is typically used after the convolutional layer to reduce the size of the feature maps. The most common method is to use the max-pooling and average pooling functions to implement a pooling layer. The fully connected layer comes after the last pooling layer, and it converts the two-dimensional feature map into a one-dimensional feature vector. To prevent overfitting, the dropout layer is used; thus, according to a probability, each neuron will be thrown out of the network at each stage of the training [45]. The batch normalization (BN) layer is typically used to normalize the data within the network and to accelerate the network training process [46]. Various types of activation functions are used in the layers, such as the Leaky Rectified Linear Unit (ReLU) and Softmax [47,48]. In the prediction stage, the “loss” function is carried out to learn the error ratio. Then, an optimization algorithm is applied for reducing the error criterion. Indeed, the optimization results are used for updating the hyperparameters. The “loss” function is a means of evaluating and representing the model efficiency in machine learning approaches [49,50].

2.3. Long Short-Term Memory (LSTM)

Recurrent neural networks (RNNs) are powerful networks that are widely used in learning from sequential data, such as text, audio, and video. RNNs can reduce the computational load of the algorithm by reducing the input data dimension and by facilitating the training process. However, as the gap between previous input information and the point of need grows, these networks encounter a lag in the learning features and perform poorly [51]. As a result, to address the shortcomings of traditional RNNs, LSTM networks have been introduced. Since prior information can affect the model accuracy, the use of LSTM has become a popular option for researchers. Unlike traditional RNNs, where the content is rewritten at every step, in LSTM, the network is able to decide whether to retain the current memory through its memory gateways. Intuitively, if an LSTM detects an important feature in the input sequence in the initial steps, it can easily transmit this information over a long period of time, thereby receiving and maintaining such long-term dependencies. Moreover, because of their memory-based design, these networks avoid gradient vanishes and the instability that plagues traditional RNNs [52,53].

3. Proposed Method

In this section, the proposed fatigue detection algorithm is provided, which is based on the CS theory and the DCLSTM network for the automatic classification of two stages of driver fatigue (a block diagram of the proposed method is shown in Figure 1). First, the acquisition of the EEG signals for the diagnosis of two-stage driver fatigue is described. Then, the preprocessing techniques performed on the recorded EEG signal are described, followed by the related details of the signal compression. Finally, the proposed DCLSTM network architecture is presented.

3.1. Acquisition of EEG Signals

Eleven graduate students (six men and five women), between 22 and 30 years of age, took part in a driving simulation experiment. It was ensured that all participants had a driver’s license, and that, up until the experiment, none of them had experienced driving in a driving simulator. All of the participants in the experiment were also right-handed. This experiment was carried out with the ethics code license number, IR.TBZ-REC.1399.6, in the signal processing laboratory of the Biomedical Engineering Department of the Faculty of Electrical and Computer Engineering, at the University of Tabriz. All participants were asked to confirm and sign the voluntary attendance consent form and the test requirements (no history of psychiatry, no history of epilepsy, and pretesting hair washing, enough sleep overnight, and no pretesting caffeine) before the experiment. The experiment was conducted using a G-Tec 32-Channel EEG Recorder, an MSI Laptop (Corei7 and 16 Ram), a Logitech Driving Simulator G29, a City Car Driving Simulator, and a Samsung 40-inch LCD. Figure 2 shows the recording of the EEG signal of a subject while driving in the simulator. The EEG signal recording was carried out following the international standard 10–20 electrode placement system, at the sampling frequency of 1000 Hz, and with the A1 and A2 channels as reference electrodes. Before the experiment, all participants practiced driving with the simulator to become acquainted with it and the purpose of the experiment. In order to induce mental fatigue in the drivers, the driving route in the simulator was considered a uniform highway without traffic. When the driving procedure began for 20 min, the last 3-min EEG recording was labeled as the “normal” stage. The ongoing driving process lasted until the participant’s questionnaire results (the multidimensional fatigue inventory scale [54]) showed that the subject was at the “fatigue” stage (varying from 40–100 min, depending on the participant), and the last 3-min EEG recordings were marked as the “fatigue” stage. The drivers were required to report their levels of fatigue using the Chalder Fatigue and Lee Fatigue questionnaires to confirm their fatigue [55,56]. The questionnaires included the following questions: “Do you feel tired?”; “Do you have a blurred vision problem?”; “Do you feel like you are constantly increasing your speed?”; “Do you feel out of focus?”, etc. Each question in the questionnaire had four scores, from −1 to 2. A score of −1 means “better than usual”, a score of 0 means “normal”, a score of 1 means “tired”, and a score of 2 means “very tired”. A high fatigue score indicates a high level of driver fatigue, which has been used in many recent studies to confirm fatigue [23,24,27,30]. The driving task started at 9 a.m., and only one EEG signal per day was recorded in order to ensure the same recording conditions for all of the participants.

3.2. Preprocessing

In order to remove unwanted artifacts from the recorded EEG signals, at first, a notch filter was applied to remove the 50-Hz frequency of the power supply. Second, a first-order Butterworth filter, with a frequency of 0.5 to 60 Hz, was applied to the data to capture useful information to detect driver fatigue [57]. Third, to improve the detection efficiency, the features for each participant were normalized by scaling between 0 and 1, and by applying min–max normalization. Fourth, since one of the objectives of this study is to use minimal EEG signal channels, it is necessary to identify the active electrodes to reduce the computational complexity. In accordance with [24,25,58,59], 12 electrodes, out of the 30 electrodes used for signal recording, were identified in the form of six active regions, on the basis of the electrode weights, for this purpose. Accordingly, only data from the 12 selected channels were used for the compression and data processing, and the rest of the channels were excluded from the processing. The selected electrodes, in the form of six regions, A, B, C, D, E, and F, are shown in 2D and 3D in Figure 3.

3.3. Signal Compression Based on CS Theory

This section provides a detailed description of how to compress the signal on the basis of CS theory. After preprocessing the signal, 12 selected electrodes enter the compression process. In the following, the process of the segmentation and compression of the signal, according to the block diagram of Figure 4, is described. As mentioned previously, 3 min of the recorded EEG channel is assigned to either the “normal” or the “fatigue” stages. In this case, we have two classes of data (normal and fatigue), with 180,000 samples for each channel. Then, every 3-min recording is divided into 5-s intervals of 150 segments, with an overlap of 1200 samples. Since we have 11 subjects, the dimension of the input matrix $X$, for each class (normal and fatigue), corresponding to each region (A to E), will be equal to N × S, where N = 11 × 150, and S is the raw signal dimension and is equal to n × w (n is the number of electrodes, and w is the length of each 5-s segment, and is equal to 5000). As stated in Section 3, in accordance with the CS theory, to guarantee the RIP condition, the input matrix, $X$, is multiplied by a random Gaussian matrix, Φ, with the dimension, M × N, in order to produce the compressed signal, Y. Considering the number of rows of the raw signal, $X$, as N = 11 × 150 = 1650, now the number of rows of the compressed signal, Y, is reduced to M = N × (1 − CR/100). In the next step, Y enters the proposed DCLSTM network for the selection/extraction of the discriminative features to perform the automatic classification.

3.4. The Proposed Deep Neural Network Architecture

In this section, the architecture of the proposed DCLSTM network will be introduced, which is depicted in Figure 5. To recognize the “fatigue” and “normal” stages, the proposed network consists of seven convolutional layers, three LSTM layers, and one Softmax layer (without using the fully connected layer), as follows: (a) A dropout layer; (b) A convolution layer with a nonlinear Leaky ReLU function and a max-pooling layer, followed by a dropout layer and a batch normalization; (c) The architecture of the previous step is repeated six times without a dropout layer; (d) The previous architecture’s output is connected to an LSTM layer with a nonlinear Leaky ReLU function, with a dropout layer and a batch normalization; (e) The architecture of the previous step is repeated two times; (f) The Softmax layer is used to access the outputs and calculate the scores.

Table 1. Details of the proposed DCLSTM-CS architecture.

L	Layer Type	Activation Function	Output Shape	Size of Kernel and Pooling	Strides							Number of Filters	Padding
					CR = 0	CR = 40	CR = 50	CR = 60	CR = 70	CR = 80	CR = 90
1	Convolution1-D	Leaky ReLU	(None, 250, 16)	128 × 1	20 × 1	12 × 1	10 × 1	8 × 1	6 × 1	4 × 1	2 × 1	16	yes
2	Max-Pooling1-D		(None, 125, 16)	2 × 1	2 × 1								no
3	Convolution1-D	Leaky ReLU	(None, 125, 32)	3 × 1	1 × 1							32	yes
4	Max-Pooling1-D		(None, 62, 32)	2 × 1	2 × 1								no
5	Convolution1-D	Leaky ReLU	(None, 62, 64)	3 × 1	1 × 1							64	yes
6	Max-Pooling1-D		(None, 31, 64)	2 × 1	2 × 1								no
7	Convolution1-D	Leaky ReLU	(None, 31, 64)	3 × 1	1 × 1							64	yes
8	Max-Pooling1-D		(None, 15 64)	2 × 1	2 × 1								no
9	Convolution1-D	Leaky ReLU	(None, 15, 64)	3 × 1	1 × 1							64	yes
10	Max-Pooling1-D		(None, 7, 64)	2 × 1	2 × 1								no
11	Convolution1-D	Leaky ReLU	(None, 7, 64)	3 × 1	1 × 1							64	yes
12	Max-Pooling1-D	-	(None, 3, 64)	2 × 1	2 × 1								no
13	Convolution1-D	Leaky ReLU	(None, 3, 64)	3 × 1	1 × 1							64	yes
14	Max-Pooling1-D		(None, 1, 64)	2 × 1	2 × 1								no
15	LSTM	Leaky ReLU	(None, 50)
16	LSTM	Leaky ReLU	(None, 30)
17	LSTM	Leaky ReLU	(None, 2)
18	Softmax		(None, 2)

displays the specifics of the proposed DCLSTM architecture with CS (DCLSTM-CS), such as the sizes of the filters, the layer types, and the number of filters. As shown in Table 1, the dimensions of the strides at different CRs are different.

3.5. Training and Evaluation

All of the hyperparameters for the proposed method were carefully adjusted to achieve the best convergence degree. A trial-and-error method was used to select these parameters. The numbers of the samples and the parameters for the training, evaluation, and test sets for all the active regions are also shown in Figure 6. According to Figure 6, 70% of the gathered data is randomly selected for training, 10% for validation, and the remaining 20% is selected as the test set. The 5-fold cross-validation was also performed for all of the active regions for a more detailed analysis. Figure 7 shows a graphical schematic for the 5-fold evaluation.

In the proposed network, the weights are initially assumed to be random and small, and they are then updated using the optimal hyperparameters on the basis of the RMSProp optimizer and the cross-entropy cost function shown in

Table 2. Optimal parameters selected for the proposed DCLSTM architecture.

Parameters	Search Space	Optimal Value
Optimizer	RMSProp, Adam, Sgd, Adamax, Adadelta	RMSProp,
Cost function	MSE, Cross-Entropy	Cross-Entropy
Number of convolution layers	3, 5, 7, 9, 11	7
Number of LSTM layers	2, 3, 5	3
Number of filters in the first convolution layer	16, 32, 64, 128	16
Number of filters in the second convolution layer	16, 32, 64, 128	32
Number of filters in other convolution layers	16, 32, 64, 128	64
Number of neurons in the first LSTM layer	30, 50, 70	50
Number of neurons in the second LSTM layer	30, 50, 70	30
Size of filter in the first convolution layer	3, 16, 32, 64, 128	128
Size of filter in other convolution layers	3, 16, 32, 64, 128	3
Dropout rate before the first convolution layer	0, 0.2, 0.3, 0.4, 0.5	0.2
Dropout rate after the first convolution layer	0, 0.2, 0.3, 0.4, 0.5	0.3
Dropout rate after the LSTM layers	0, 0.2, 0.3, 0.4, 0.5	0.2
Batch size	4, 8, 10, 16, 32, 64	10
Learning rate	0.01, 0.001, 0.0001	0.0001

. In designing the proposed network architecture, we used different types of optimizers, and different numbers and filter sizes, and we achieved the optimal values for the parameters of the proposed architecture, which are shown in Table 2.

4. Results and Discussion

The simulation results, comparisons with previous studies, and an intuitive evaluation of the proposed method are presented in this section.

4.1. Simulation Results

The simulation results of the proposed DCLSTM-CS algorithm for the automatic detection of driver fatigue are discussed in this section. The computer specifications used to simulate the proposed method are as follows: 32 GB of RAM, a Core i9 CPU, and an RTX 2070 Graphics card. The different parameters for the proposed DCLSTM network were fine-tuned using the trial-and-error method. As is shown in Figure 8, we used different numbers of convolution layers (three to eleven layers) in the proposed network design, while seven layers of convolution is considered as the optimum solution (in terms of accuracy and speed). As can be seen from Figure 8, by increasing the number of convolution layers (above seven), the classification accuracy remains almost constant, and the training time increases accordingly. As mentioned before, in our proposed algorithm, the compressed signal is used as the DCLSTM network input, for which

Table 3. Classification accuracies obtained for different regions in different CRs.

Region	A (%)	B (%)	C (%)	D (%)	E (%)	F (%)
ACC for NC	98.7	96	98.2	97.6	99.1	93.5
ACC for CR = 40	96.3	95.15	95.9	94.9	95	91.3
ACC for CR = 50	96.1	95	95.7	94.6	94.8	89.9
ACC for CR = 60	95.5	94.09	94.8	94.1	94.6	89.2
ACC for CR = 70	94.8	93.94	94.2	93.8	94.4	88.8
ACC for CR = 80	94.1	93.48	93.6	93.1	94.4	88.6
ACC for CR = 90	93.7	92.72	91.1	91.8	92	86.1

shows the accuracy obtained on the validation data. As is shown in Table 3, the performance of the proposed algorithm for the noncompression (NC) mode, where the raw signal is directly fed into the DCLSTM network, is given, as well as for the compression mode, where the CR is set to 40, 50, 60, 70, 80, and 90. As is clear from the NC mode, the final accuracy for all regions is over 93%. Moreover, as can be seen, the accuracy of Region E, which includes a single channel, is approximately 98%. In addition, as is clear, even at CR = 90, the accuracy of the proposed network for all regions, except for Region F, is still above 90%. Figure 9 shows the loss of the proposed network for six active regions, where CR = 40. As can be seen from Figure 9, increasing the number of iterations leads to a decrease in the losses for all regions, reaching a steady-state value at approximately 300 iterations. Figure 10 shows the classification accuracy of the proposed network where CR = 40, and, as can be seen, the classification accuracies for Regions A, B, C, D, E, and F reach 96.3, 95.15, 95.9, 94.9, 95, and 91.3%, respectively, at about 310 iterations. The confusion matrix for the proposed DCLSTM-CS is provided in Figure 11, for all regions at CR = 40. As can be seen, only 21 samples were misdiagnosed as “fatigue” stage in Region E, indicating that the proposed network performs well. For further analysis, Figure 12 shows the performances (sensitivity, precision, specificity, F-score, kappa, accuracy, and training time) of the proposed network for the single-channel region (E) at different CRs, as well as the NC mode. As can be seen from Figure 12, the network training time decreases as the CR increases; however, the network performance for CR = 90 is still higher than 90%, which indicates a promising performance for the proposed network in single-channel driver-fatigue-detection scenarios. Furthermore, Figure 13 shows the T-Sen graph for the raw signal and the LSTM2 layers for the single-channel region (E) at CR = 80, as well as the NC mode. As can be seen from the last layer in the different CRs, almost all of the samples are separated for the evaluation set in each of the CRs, indicating the desirable performance of the proposed method for the two-stage classification of driver fatigue. As a result, on the basis of the questionnaire, the model calculation results are consistent with the actual fatigue of the participants. Therefore, the proposed method can detect driver fatigue effectively. To further research the efficacy of the proposed method, Figure 14 shows the classification accuracy obtained for each fold in all the selected regions at different CRs. As is shown in Figure 14, the accuracy obtained for each fold at different compression rates is approximately higher than 90%, indicating that the overfitting phenomenon did not occur in different folds.

4.2. Comparison with the State-of-the-Art Methods

The proposed method was evaluated in a variety of ways. First, it was compared with the common and popular methods used to diagnose driver fatigue, and second, it was compared with the most recent studies. In order to demonstrate the performance of the proposed algorithms, based on DCLSTM and DCLSTM-CS, the two-stage classification of driver fatigue based on Region E (single-channel) was simulated using four existing popular networks. The networks to be compared included MLP [60], DBM [61], SVM [62], and Fully CNN (FCNN) [63], and they are based on feature learning from raw data, as well as manual feature extraction (for the manual features, the mean, minimum, crest factor, skewness, variance, maximum, and kurtosis were also used), which have recently been widely used in driver-fatigue detection studies. For the FCNN, the proposed network architecture is considered in Figure 5, without the use of LSTM. As the kernel function of the SVM, the Gaussian radial basis function (RBF) was used, and to optimize the kernel parameters, the grid search method was used. In addition, the number of hidden layers and the learning rate for the MLP and DBM networks is 3 and 0.0001, respectively. The accuracies obtained, on the basis of feature learning from the raw data and manual features, are presented in

Table 4. Accuracies of the proposed DCLSTM and DCLSTM-CS methods, as well as of the FCNN, DBM, SVM, and MLP methods for both feature learning and manual feature extraction scenarios.

Methods	Feature Learning from Raw Data (%)	Manual Features (%)
DCLSTM	98	80
DCLSTM with CS	95	76
FCNN	88	80
DBM	85	77
SVM	72	78
MLP	70	79

. As is shown in this table, applying feature learning in deep networks (FCNN, DBM, DCLSTM, and DCLSTM-CS) leads to a significant improvement in terms of the accuracy, compared with the manual feature extraction approach. As it is seen in Table 4, the proposed DCLSTM and DCLSTM-CS present the highest accuracies, compared to the other deep and traditional approaches (i.e., SVM and MLP). The accuracies of the proposed DCLSTM and DCLSTM-CS (for CR = 40), together with FCNN, DBM, SVM, and MLP, based on feature learning, are shown Figure 15. The accuracies of the proposed DCLSTM, DCLSTM-CS, and the FCNN, DBM, SVM, and MLP reach 98, 95, 88, 85, 72, and 70%, respectively, after 400 iterations. As is seen, the accuracies of the proposed networks are significantly higher than the existing methods, while having faster convergence rates. As is shown in Figure 15, although the proposed DCLSTM network has higher accuracy than its compressed mode (DCLSTM-CS), the accuracy achieved by DCLSTM-CS is still remarkably higher that the existing methods (about 95%), and it is more computationally efficient compared to DCLSTM. In addition, for a thorough comparison of the proposed methods with the existing state-of-the-art methods, a performance of all the methods was examined in a noisy environment. For this purpose, a white Gaussian noise of SNR, from −4 to 20 dB, was added to the EEG signals as measurement noise, and the classification accuracies for all the methods are shown in Figure 16. As can be seen, the proposed DCLSTM and DCLSTM-CS algorithms are quite robust to the measurement noise over a wide range of SNRs, such that the classification accuracy is still above 90%. A number of studies have been conducted in recent years in the field of the automatic detection of driver fatigue. The best results presented in these studies are shown in

Table 5. Accuracies of the proposed networks compared with existing state-of-the-art methods.

Research	Feature Method	Highest Accuracy (%)
Corea et al. [14]	Multimodal Analysis	83
Xiong et al. [15]	AE and SE	90
Chai et al. [16]	Entropy Rate Bound Minimization Analysis	88.2
Zhang et al. [17]	Entropy and Complexity Measure	96.5
Yin et al. [18]	FE	95
Ko et al. [19]	FFT	90
Wang et al. [20]	PSD	83
Mu et al. [21]	EEG Frequency Ratio	85
Nugraha et al. [22]	Emotive	96
Karuppusamy et al. [28]	DNN	73
Liu et al. [30]	Deep Transfer Learning	93
Hu et al. [23]	Multiple Entropy	97.5
Min et al. [24]	Multiple Entropy	98.3
Cai et al. [25]	Horizontal Visibility Graph	98
Luo et al. [26]	Adaptive Scaling Factor and Multiple Entropy	95
Jiao et al. [29]	CWT-GAN-LSTM	98
Gao et al. [27]	CNN	95
P-M (DCLSTM)	DCLSTM	99.23
P-M (DCLSTM with CS)	DCLSTM-CS	96.36

and are compared to the proposed algorithms. As is shown in Table 5, the proposed methods have the best performance when compared to previous studies.

Despite the good performance of the proposed method, this study, as with others, has some limitations:

• The statistical population considered in this study is limited and should be expanded;
• Wet electrodes were used to record the signal, which can be problematic in real-world scenarios, so the performance of dry electrodes must also be evaluated;
• In this study, the scenarios considered for signal recording were only related to passive fatigue, whereas, in real scenarios, fatigue due to stress and active fatigue must also be examined.

5. Conclusions

Driver fatigue has now become one of the most important factors in car accidents, and it has attracted the attention of various communities and car manufacturers interested in the development of automatic driver-fatigue-detection methods. Among the studies to automatically detect driver fatigue on the basis of EEG signals, most of the research focuses on increasing the classification accuracy, regardless of the involved computational load; however, the design of real-time driver-fatigue-detection systems requires an algorithm that uses minimal physiological channels, as well as one that has a high speed and accuracy. This study presents an automatic algorithm that can be used in real-time driver-fatigue-detection systems. In the proposed algorithm, the recorded EEG signals are compressed on the basis of CS theory. Then, in order to implement the automatic feature selection/extraction and classification, compressed EEG signals are fed to the proposed DCLSTM neural network. The proposed network consists of seven CNN layers and three LSTM layers, which are capable of taking full advantage of the CS and deep learning to achieve a relatively high performance with a small amount of EEG data. As a result, with a CR = 40, the accuracy of classification is still above 90% for all the active regions (A to F). Because of the fact that the proposed method uses minimal EEG electrodes to detect driver fatigue, it is also very accurate and has a low sensitivity to environmental noise. As a result, the proposed algorithm can be used to design real-time automatic driver-fatigue-detection systems with high accuracy and speed. In the future, transfer learning networks, such as ResNet, Inception, and VGG, can be used in combination with CS to reduce the training time. GANs can also be used for data augmentation and to avoid overfitting problems, and their performance could be compared with the classical data augmentation used in this study.