Next Article in Journal
Anti-Aliasing Attention U-net Model for Skin Lesion Segmentation
Next Article in Special Issue
On the Use of a Convolutional Block Attention Module in Deep Learning-Based Human Activity Recognition with Motion Sensors
Previous Article in Journal
Optical Coherence Tomography and Optical Coherence Tomography Angiography in Pediatric Retinal Diseases
Previous Article in Special Issue
Assessing Passengers’ Motion Sickness Levels Based on Cerebral Blood Oxygen Signals and Simulation of Actual Ride Sensation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Study of Motion Sickness Model Based on fNIRS Multiband Features during Car Rides

Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
*
Author to whom correspondence should be addressed.
Diagnostics 2023, 13(8), 1462; https://doi.org/10.3390/diagnostics13081462
Submission received: 27 February 2023 / Revised: 6 April 2023 / Accepted: 14 April 2023 / Published: 18 April 2023
(This article belongs to the Special Issue Wearable Sensors and Artificial Intelligence for Ergonomics)

Abstract

:
Motion sickness is a common physiological discomfort phenomenon during car rides. In this paper, the functional near-infrared spectroscopy (fNIRS) technique was used in real-world vehicle testing. The fNIRS technique was utilized to model the relationship between changes in blood oxygenation levels in the prefrontal cortex of passengers and motion sickness symptoms under different motion conditions. To enhance the accuracy of motion sickness classification, the study utilized principal component analysis (PCA) to extract the most significant features from the test data. Wavelet decomposition was used to extract the power spectrum entropy (PSE) features of five frequency bands highly related to motion sickness. The correlation between motion sickness and cerebral blood oxygen levels was modeled by a 6-point scale calibration for the subjective evaluation of the degree of passenger motion sickness. A support vector machine (SVM) was used to build a motion sickness classification model, achieving an accuracy of 87.3% with the 78 sets of data. However, individual analysis of the 13 subjects showed a varying range of accuracy from 50% to 100%, suggesting the presence of individual differences in the relationship between cerebral blood oxygen levels and motion sickness symptoms. Thus, the results demonstrated that the magnitude of motion sickness during the ride was closely related to the change in the PSE of the five frequency bands of cerebral prefrontal blood oxygen, but further studies are needed to investigate individual variability.

1. Introduction

Motion sickness is a common symptom that includes dizziness, fatigue, loss of appetite, gastrointestinal distress, and vomiting [1]. Its duration can vary from a few minutes to several hours [2]. Although motion sickness usually does not lead to serious harm, severe cases can cause discomfort, including headaches and cold sweats, which may negatively impact one’s health [3]. The direct causation of motion sickness is thought to come from the conflict between human expectations and measurements of spatial orientation rather than a specific sensory modality [4]. The prefrontal cortex (PFC) plays an essential role in the level of driving, as indicated by differences in brain activity observed in fNIRS studies [5]. In addition, the altered blood supply to the local area is an important direct manifestation of neuronal dysfunction [6]. Therefore, the study of blood supply to the prefrontal region of the brain is of great importance.
Since the 1970s, researchers have been studying motion sickness by analyzing physiological signals with methods such as the electroencephalogram (EEG), the electrocardiogram (ECG), heart rate variability (HRV), and the electrogastrogram (EGG) [7]. Zhao, L. et al. investigated the correlation between automatic and active driving by collecting motion sickness scores and EEG signals from subjects during automatic and active driving. The results showed that subjects had higher motion sickness scores on average under automatic driving than under active driving. As the degree of motion sickness increased, the frequency of the center of gravity of the brain motor center (FC2, Cz) theta wave power spectral density increased [8]. Lim, H.K. et al. evaluated the reliability of using the EEG to evaluate motion sickness and identified specific waves and regions [9]. Li, Z. et al. further demonstrated the validation of the mean EEG frequency of theta band as an indicator of motion sickness [10]. Ji-Un, H. et al. fused five physiological signals—EEG, ECG, respiration (RESP), photoplethysmography (PPG), and galvanic skin response (GSR)—to construct a motion sickness classification model based on real-world vehicle testing [11]. Although the use of various physiological signals is a more objective response to the level of motion sickness, it has been suggested that HRV and GSR may be controversial indexes that need to be reassessed before they can be used again for the assessment of motion sickness [12]. In addition, Tan, R. et al. investigated motion sickness detection using physiological signals in on-road driving scenarios [13].
Motion sickness is thought to be caused by a conflict between the vestibular, visual, and other proprioceptive systems, which are involved in the perception of motion [14]. The prefrontal area of the brain is important to human perception [15]. Therefore, the fNIRS technique was adopted in this study.
fNIRS is a noninvasive, real-time technique for monitoring metabolic activity in living organisms [16]. It can measure blood oxygenation and related parameters in biological tissues [17]. The fNIRS technology detects changes in the concentration of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) in tissues by measuring the absorption of near-infrared light in the near-infrared wavelength range, thus enabling the monitoring of the functional activity of biological tissues [18]. In recent years, the fNIRS technique has been widely used in medicine, exercise physiology, neuroscience, nutrition, and bioengineering [19]. Compared with other traditional physiological monitoring techniques, fNIRS has advantages such as high spatial resolution ranging from 1 to 3 centimeters, temporal resolution ranging from 100 milliseconds to 1 s, and ease of operation [20]. This study aims to investigate the potential of fNIRS technology for the study of motion sickness with the goal of improving our understanding, diagnosis, and treatment of this condition.
Research on motion sickness and cerebral blood oxygen concentration has made some progress nationally and internationally. Zhang et al. (2019) showed that the Brodmann area of the brain associated with motion sickness reacts differently under straightforward working conditions. The findings contribute to the study of motion sickness from a brain perspective [7]. Zhang’s team again designed environments for straight and curved driving conditions in 2020, performed driving maneuvers, and recorded driver brain activity to investigate the mechanisms correlating motion sickness with cortical activity. The findings suggest that the emergence of the motion sickness response is associated with the occipital lobe and propose a new research method to investigate the correlation mechanism between motion sickness and cortical activity [7]. The study by Golding, J.F. et al. has increased knowledge of the environmental and behavioral conditions that trigger and prevent motion sickness as well as the sensitivity of vestibular disorders to motion sickness and the associated brain mechanisms [21]. Gavgani, A.M. et al. reported a significant increase in HbO concentrations in channel 2/15 in the parieto-temporal regions of both hemispheres in subjects presenting with motion sickness. Increased HbO concentrations were associated with increased nausea and motion sickness symptoms. The study suggested that motion sickness leads to complex changes in cerebral blood flow, with increased perfusion in some cortical regions but decreased overall cerebral perfusion [6].
The purpose of this study is to establish a model that examines the relationship between motion sickness and prefrontal cerebral blood oxygen concentration. This is accomplished by using fNIRS technology to measure blood oxygen concentration in the prefrontal cortex as well as by collecting data on motion sickness symptoms during real-world vehicle testing supported by SAIC Group.

2. Materials and Methods

2.1. System Architecture Diagram

The architecture of the entire system is shown in Figure 1. This study aimed to develop a classification model for the degree of motion sickness. The entire process consists of 4 main steps. First, the cerebral blood oxygen signal was preprocessed, including mean filtering, median filtering, and motion artifact suppression (MAS), to obtain the preprocessed data. Second, one of the most significant feature channels was obtained by PCA. Third, wavelet decomposition was performed on the channel to obtain the data of frequency bands I–V. The features of PSE are extracted based on different frequency bands. Fourth, the data were fed into SVM for machine learning by combining a 6-point subjective evaluation of the motion sickness scale. Through validation, a motion sickness model was obtained to achieve the classification of the degree of motion sickness.

2.2. fNIRS Signal and Preprocessing

fNIRS takes advantage of the better transmittance of NIR light in tissues by placing multiple light sources and light detectors on the scalp to record the intensity changes of NIR light as it passes through the scalp, skull, and brain tissue. The operating principle of fNIRS is based on two different near-infrared light absorption spectra, one at around 760 nm, absorbed by HbO and HbR, and the other at around 850 nm, absorbed only by HbO. Figure 2 shows a subject wearing the fNIRS device, and the magnified portion on the right side of Figure 2 displays the measurement principle of the fNIRS device. Specifically, the red section on the left represents the “light source,” which includes two types of near-infrared light in the 730 nm and 850 nm wavelength bands, passing through the brain tissue of the subject’s forehead and finally being received by the light-intensity sensor, or “Detector”. By simultaneously measuring the intensity changes of NIR light at these 2 wavelengths and using the Lambert–Beer law formula [19], changes in the concentrations of HbO and HbR can be calculated, thus reflecting changes in the level of metabolic activity in brain regions.
Δ I 1 I 1 Δ I 2 I 2 = ϵ H b R Δ C H b R ϵ H b O Δ C H b O
where C H b O and C H b R represent changes in the concentrations of HbO and HbR, respectively. ϵ H b O and ϵ H b R denote the molar absorption coefficients of the HbO and HbR, respectively, and the variables I 1 and I 2 denote the variation of the NIR light intensity at 2 wavelengths. Based on Equation (1) above [22,23], the amount of change in the concentration of HbO and HbR, and consequently the concentration of oxyhemoglobin, can be calculated by measuring the change in NIR light intensity at 2 wavelengths, I 1 and I 2 .
The first 5 min of data were excluded from this study. A series of pretreatment steps were used in this study. First, the raw light-intensity signal of each channel was mean filtered (MeanFilt) and median filtered (MedFilt) to reduce the effect of high- and low-frequency noise. The main frequency components of the signal were also retained to better reflect the actual variation of the physiological signal.
In addition, when subjects are performing fNIRS acquisitions, movements of their head or body cause relative changes in the distance between the optical probe and the area being measured. These changes lead to motion artifacts in the fNIRS signal. To solve this problem, this study uses discrete wavelet transform to perform motion artifact suppression [24] on the 6-channel fNIRS signal, thresholds the wavelet coefficients obtained after several decompositions, and takes out the wavelet coefficients that exceed the thresholds and reconstructs them to achieve the removal of motion artifacts. This method can effectively improve the quality and reliability of the signal and provide a more accurate basis for subsequent data analysis [24]. As shown in Figure 3a–d, the fNIRS data preprocessing procedure included MeanFilt, MedFilt, and MAS to obtain the preprocessed data, which was then used for subsequent analyses. In addition, to more clearly observe the effects of data preprocessing, Figure 3e shows the light intensity signal diagrams of MeanFilt, MedFilt, and MAS in the same figure sequentially.

2.3. Channel Dimensionality Reduction Based on PCA

The signal-to-noise ratio of HbO is higher than that of HbR, which is more sensitive. In addition, when the two signals are combined for analysis, this does not improve the accuracy of the results much. Thus, this study was conducted to construct a motion sickness model by analyzing the changes of HbO content in the prefrontal area of the brain [25]. The fNIRS device has a total of 6 channels. The relative amount of change in HbO can be calculated by acquiring the change in absorbed light intensity based on the Lambert–Beer law algorithm.
The fNIR103E device has 6 sampling channels, and the six channels are correlated with each other and have some redundancy. Therefore, this study uses PCA [26] to downscale the data of the original 6 channels to obtain the single-channel information with the most significant features [22].
PCA is a common data dimensionality reduction technique. Its basic idea is to project the original data onto a new set of coordinate systems, which can be obtained as PC1 and PC2, the first and second principal components obtained in PCA, respectively. PC1 and PC2 are the new variables obtained from the original variables after linear transformation. PC1 is the direction of maximum variance obtained by linear combination of the original variables, that is, the direction of maximum variance of the data. PC2 is the direction orthogonal to PC1 and is the direction of maximum variance of the residuals in the direction of PC1, that is, the linear combination of maximum variance outside the first principal component.
In this study, the scikit-learn library in Python was used for PCA analysis, and the number of retained principal components was selected based on the cumulative variance contribution and Kaiser criterion. Usually, the first two principal components will explain most of the variation of the data. Therefore, the channel corresponding to the maximum value can be taken as the principal component channel.
As shown in Figure 4, the projection values for each channel on PC1 and PC2 are calculated, with PC1 and PC2 being the first and second principal components obtained from Subject 1’s 6 samples. This study used PCA to downscale the original 6-channel data to obtain the single-channel information with the most significant features. By normalizing the projection values on PC1 and PC2, the relative size and orientation of each channel were determined. Both PC1 and PC2 have values ranging from −0.6 to 0.68, and the most significant channels in each segment are shown as red circles in Figure 4.

2.4. Feature Extraction of Multiband PSE

For cerebral blood oxygen signals, different frequency bands originate from different physiological activities, and each frequency band has a specific physiological meaning [27,28]. Currently, the cerebral blood oxygen signal is a time series signal in the main range of 0–2 Hz. These signals can be divided into 5 frequency bands and used for the study of motion sickness classification. The specific frequency bands are divided as shown in Table 1. These frequency bands have an important role in the study of motion sickness [29,30].
According to Nyquist’s sampling theorem, the sampling frequency should be greater than 2 times the highest frequency of signal extraction, that is, f s . m a x > 2 f m a x [31]. Only in this case can the sampled signal retain the information in the original signal intact; otherwise, sampling distortion will occur [32].
Since the cerebral blood oxygen signal collected by the NIR experimental equipment is in the time domain, converting the time domain signal to the frequency domain signal is the best way to gain insight into the relationship between blood oxygen changes in the prefrontal area and the degree of motion sickness [33,34]. In this study, the single channel basis with the most significant correlation features screened by PCA in the previous section was selected, and wavelet decomposition was performed on this channel. The cerebral blood oxygenation signal is a nonstationary signal, which cannot be analyzed by the conventional Fourier Transform [35]. The Short Time Fourier Transform [36], on the other hand, has a single resolution due to its fixed window function. To clarify, the "infinite-length trigonometric function" in the Short Time Fourier Transform can be replaced with a "finite decayable wavelet basis" to achieve time-frequency conversion. This enables the signal features to be adjusted to the appropriate window size, resulting in enhanced resolution.
Wavelet transform has multiscale and multiresolution features, which can gradually focus and amplify the signal to be analyzed and can distinguish local features [36]. The signal f ( t ) is represented or approximated by a certain function, which is composed of the basic wavelet basis function ψ ( t ) scaled and translated by different scales. Let ψ ( t ) ϵ L 2 ( R ) , when ψ ( ω ) satisfies the allowable condition [37], be:
C ψ = ψ ( ω ) ω <
Then, ψ(t) can be called a fundamental wavelet or mother wavelet function. The wavelet sequence is obtained by stretching and translating it [37]:
ψ a 0 b t = 1 a ψ ( t b a )
In the equation, a and b are the scaling parameter and the scale parameter, respectively. Since the cerebral blood oxygen signal consists of discrete sampling points, a and b are then discretized. Take a = a 0 m , b = n b 0 a 0 m , where m and n are integers, a 0 > 1 is a fixed scaling step, b 0 > 0 , and is related to the specific form of the mother wavelet. Thus, the discrete wavelet function can be defined as [37]:
ψ m 0 n t = a 0 m 2 ψ t n b 0 a 0 m a 0 m = a 0 m 2 ψ ( a 0 m t n b 0 )
The wavelet basis function used in this study is Daubechies wavelet because of its excellent properties in orthogonality, time-frequency tight support orthogonality, etc. To obtain a better signal decomposition effect, db5 with 5th-order vanishing moment in Daubechies wavelet series was chosen as the wavelet basis function in this study. In order to determine the number of layers of decomposition, it is necessary to combine the sampling frequency and the range of frequency bands. According to the previously mentioned range of frequency bands I–V, the PSE features of 5 frequency bands need to be extracted in this study, and the sampling frequency of the fNIRS equipment is 10 Hz, so 9 layers of wavelet decomposition are selected. Figure 5 shows the number of wavelet decomposition layers and the corresponding frequency band ranges.
As shown in Figure 5, after wavelet decomposition, the timing diagram corresponding to the frequency bands I–V can be obtained, and the 9-layer wavelet decomposition are obtained as approximation coefficients (a1–a9) and detail coefficients (d1–d9). The detail coefficients, which contain the high-frequency part of the original signal, can provide information about signal variations and local features. Wavelet decomposition of the cerebral blood oxygen signal yields the detail coefficients d for different frequency bands, where the value of detail coefficient d3 corresponds to frequency band I (0.6–2.0 Hz), the value of d5 corresponds to frequency band II (0.145–0.6 Hz), the value of d6 corresponds to frequency band III (0.052–0.145 Hz), the value of d8 corresponds to frequency band IV (0.021–0.052 Hz), and the value of d9 corresponds to band V (0.0095–0.021 Hz). By analyzing the time domain diagram of the 5-band signal, an in-depth understanding of the characteristics and various patterns of the signal can be obtained. Effective features can also be extracted. The relationship between signals in different frequency bands and physiological activities can be further explored to provide a theoretical basis and reference for related research.
So far, the wavelet transform method has obtained the cerebral blood oxygen timing maps of 5 frequency bands. Next, we select the PSE of the 5 frequency bands as the features of the motion sickness classification model. PSE is a method for calculating the spectral features, and by calculating the spectral density of cerebral blood oxygen signals, we can obtain information such as the complexity and equilibrium of cerebral blood oxygen signals. In this study, PSE was selected as the feature basis because it has better robustness and reliability [38], can better reflect the characteristics and differences of signals in different frequency bands [39,40], can be used as a new nonlinear measure of signal complexity, and can be used for feature extraction and classification of signals [40] to provide effective feature information for the establishment of motion sickness classification models.
For time domain discrete sequence x n , n = 0,1 , , N 1 , the PSE is denoted as [41]:
S x = 1 N x ( k ) 2 , k = 0,1 , 2 , , N 1
The energy is conserved during the transformation of the signal from the time domain to the frequency domain, that is [41],
n = 0 N 1 x n 2 = k = 0 N 1 S x ( k ) 2
Considering S x k , k = 1,2 , , 3 , N as an energy division of the signal x t in the instantaneous frequency domain space, the PSE of the signal can be calculated as [41]:
F S E = k = 1 N p k l o g 2 p k
p k = S x ( k ) / k = 1 N S x ( k )
where p k denotes the percentage of the kth power spectrum in the whole power spectrum and also denotes the frequency of the kth frequency corresponding to the amplitude occurrence. The normalized PSE is [41]:
F P S E = F S E l o g 2 N
The above Equations (5)–(9) can be derived from the PSE corresponding to the Ⅰ–V frequency band. So far, the feature extraction of the PSE of multiple frequency bands has been completed, providing the feature input of data for the following construction of the motion sickness classification model, realizing the automatic identification and evaluation of the degree of passenger motion sickness.

2.5. Motion Sickness Model Construction Combined with Subjective Evaluation

In this study, subjects were counted every 5 min on a 6-point scale of subjective measures of motion sickness [42] in real-world vehicle testing. The 6-point motion sickness scale, as shown in Table 2, includes “no motion sickness at all,” “slight motion sickness,” “some motion sickness,” “much motion sickness,” “extreme motion sickness,” and “unbearable motion sickness.” Each degree was scored separately, corresponding to a scale of 0–5. For statistical purposes, a score of 0–1 is considered as no motion sickness (labeled as 0 in SVM), and a score of 2–5 is considered as motion sickness (labeled as 1 in SVM) [43]. Therefore, this study belongs to a typical dichotomous model for motion sickness discrimination.
SVM [44] is a commonly used machine learning algorithm for binary and multiclassification tasks. The basic principle is to find an optimal hyperplane (decision boundary) in a high-dimensional space to separate different classes of data [45]. Specifically, for the binary classification problem, the SVM tries to find an optimal hyperplane such that all data points belonging to Category 1 lie on one side of the hyperplane and all data points belonging to Category 2 lie on the other side.
The input feature of this study is the PSE of frequency bands I-V, and the output label is the degree of motion sickness of passengers. Specifically, the multiband PSE feature vector of each subject is used as a training sample, and its corresponding motion sickness degree is used as a label.
The decision function of SVM can be expressed as [46]:
f x = s i g n ( w T x + b )
where x is the feature vector of the input samples, w is a weight vector, and b is a bias term. The objective of the SVM is to maximize the interval between the hyperplane and the nearest sample point, that is, to minimize the norm of w , while making all samples satisfy the classification requirements. Specifically, the SVM aims to make the objective function minimized [46]:
m i n ω 1 2 ω 2 2 ξ i + C i = 1 m ξ i
where C is a penalty coefficient to control the balance between the classification error and the decision boundary, and ξ i is a slack variable to deal with non-distinguishable sample points.
SVM makes a linearly indivisible problem linearly divisible in a high-dimensional space by mapping the input samples into the high-dimensional space. Specifically, SVM uses a kernel function to compute the inner product of the input samples in the high-dimensional space to obtain a nonlinear decision boundary.
In this study, the Radial Basis Function (RBF) is used as the kernel function to calculate the inner product of the input samples in the high-dimensional space by using the following equation [46]:
K x , x = e x p ( γ x x 2 )
where γ is a parameter that controls the shape of the RBF kernel function.
The SVM algorithm is used to construct a motion sickness binary classification model using the PSE feature vector of multiple frequency bands as the input and the motion sickness degree as the output label.

3. Passenger Motion Sickness Experimental Design

The test is aimed to collect data from 13 subjects who had no medical history of relevant diseases and did not use any psycho-stimulating substances such as coffee, strong tea, or drugs within 24 hours before the experiment. The subjects included 6 males and 7 females with an age range of 26 to 32 years.
The experiment was conducted using actual road tests to simulate real-world vehicle riding. The experimental design scheme is shown in Figure 6. During driving, the subjects were required to sit still in their seats for 5 min, followed by a 30-min riding session with a 20-min break in between for the next set of experiments.
In order to reduce the influence of external factors on the experimental results, only 1 subject was arranged for each day, and the experimental environment was chosen as “Outer Ring-S7-Baoan Highway–Hutai Road Loop” in Baoshan District, Shanghai, with a total distance of about 12.1 km. To avoid the relief of motion sickness by frequent traffic light starting and stopping, this study conducted the test between 2:00 p.m. and 4:00 p.m., when traffic light is light. The experimental vehicle chosen was the F7 series of new energy vehicles provided by SAIC. Figure 7 shows the test vehicle, route, and pictures of the left, front, and right views of one subject for the real-world vehicle test experiment. In addition, we also recorded some underlying assumptions during the experiment, such as fixed speed intervals, driving time, and driver experience, which can be provided to readers if needed.
During driving, the lead driver adjusts the power response and regen (regenerative braking) intensity of the vehicle; subjects are not informed about these adjustments. Subjects fill out a 6-point scale for subjective evaluation of motion sickness every 5 min to record the degree of motion sickness at the current time point. Consent was obtained from the subjects before the experiments were conducted, and the ethical requirements for human subjects were strictly observed during the experiments.
The study evaluated the classification model using a 5-fold cross-validation approach by dividing the dataset into 5 parts, selecting 4 of them for training and the remaining 1 part for testing each time and repeating the process 5 times, selecting a different test and training set each time. The confusion matrix can be used to compare the differences between the model predictions and the true results [47]. As shown in Figure 8a–d, the confusion matrix illustrates the classification results of dizziness severity for 13 subjects. The rows of the matrix represent the true classes of the subjects, while the columns represent the predicted classes. The diagonal elements indicate the number of subjects whose predicted dizziness severity matches their actual severity, while the off-diagonal elements indicate the misclassified subjects. It is worth noting that the accuracy rate for the 13 subjects varies, with the highest accuracy being 100% and the lowest being for Subject 2, with only a 50% accuracy rate. In this motion sickness model, the confusion matrix divides the true and predicted outcomes into 4 categories, true positive (T.P.), false positive (F.P.), true negative (T.N.), and false negative (F.N.), respectively. True positive refers to the number of samples where the model correctly predicts positive cases, false positive refers to the number of samples where the model incorrectly predicts negative cases as positive cases, true negative refers to the number of samples where the model correctly predicts negative cases as negative cases, and false negative refers to the number of samples where the model incorrectly predicts positive cases as negative cases. These results can be represented in the form of a matrix. As shown in Figure 9, a confusion matrix is plotted for 78 sets of samples, where 32 samples are correctly classified as negative cases (no motion sickness) and 36 samples are correctly classified as positive cases (motion sickness). Of these, 68 samples were correctly classified, resulting in an accuracy rate of 87.18%. T.P., T.N., F.P., and F.N. denote the number of true, true negative, false positive, and false negative cases, respectively, with a recall metric of 0.9143 and an F1 value of 0.8852.

4. Discussion

This study has some limitations that should be acknowledged. First, data collection was performed through real-world road tests, which may be affected by various factors such as driving route, speed, and environment. These factors may lead to variations in the collected data and limit the generalizability of our findings. Future studies could consider using simulators or virtual reality technologies to control for these variables and improve the validity of the data.
Second, our study employed a small sample design, which may limit the statistical power and generalizability of our conclusions. Future research could adopt larger sample sizes and multicenter studies to enhance the reliability and generalizability of our findings.
Finally, while PCA and wavelet decomposition were used to extract features from the prefrontal blood oxygen signals in our study, other more advanced algorithms could be explored for further data analysis, such as deep learning methods and independent component analysis. These advanced algorithms may provide more accurate and reliable results for future studies in this field.
Despite these limitations, our study contributes to the understanding of motion sickness and provides insights for developing effective interventions to reduce its incidence. Future studies could build upon our findings and address these limitations to further advance this field of research.

5. Conclusions

This study intended to investigate the relationship between prefrontal blood oxygen signal and motion sickness during riding. It was found that there was a significant correlation between the PSE characteristics of the subjects’ brain prefrontal in five frequency bands and the subjects’ degree of motion sickness. Further information was extracted through PCA and wavelet decomposition. PCA helped this study to extract the most significant features from the six channels, and wavelet analysis helped us to determine the appropriate number of stratifications. The HbO values of different frequency bands were also analyzed. In this study, data were collected from 13 subjects using BIOPAC’s fNIRS cerebral oximeter with real-world road tests. The results of the study suggest that the magnitude of motion sickness is strongly correlated with changes in the entropy of the PSE of the five frequency bands of cerebral prefrontal blood oxygen. To further investigate the relationship between each frequency band and the classification accuracy, we used correlation analysis. The results indicated that there was a certain degree of correlation between the PSE characteristics in frequency bands I, III, and IV and the subjective evaluation of motion sickness, with correlation coefficients of 0.6, 0.2, and 0.19, respectively. This finding could be beneficial for understanding the development of motion sickness and exploring potential methods to alleviate it during car rides.

Author Contributions

All authors have made substantial contributions to the manuscript. Conceptualization, B.R.; methodology, B.R., W.G., and Q.Z.; data acquisition, W.G. and Q.Z.; data analysis and interpretation, W.G. and Q.Z.; drafting of the manuscript, B.R., W.G., and Q.Z.; critical revision of the manuscript, B.R., W.G., and Q.Z.; funding acquisition, B.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China (Grant No. 51775325); the Joint Funds of the National Natural Science Foundation of China (Grant No. U21A20121); the Young Eastern Scholars Program of Shanghai (Grant No. QD2016033); and the Hong Kong Scholars Program of China (Grant No. XJ2013015).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Academic Ethics Committee of Shanghai University (2022-011), data of approval: 22 February 2022.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

All data included in this study is available upon request by contact with the corresponding author. The data is not publicly available because of ethical restrictions.

Acknowledgments

The authors appreciate the project supporting company: SAIC Motor R&D Innovation Headquarters, SAIC Motor Corporation Limited, Shanghai 201804, China; supportive colleagues Ronghui Tong, Wenfa Luo, Bin Wang, Yuanfei Hei, and Shuang Wang. We also gratitude to the laboratory students Qibing Li, Xiaoyuchen Wang, Pengyu Ren, and Lin Wang for the participation in the real car test for motion sickness.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Thompson, D.J.; Kouroussis, G.; Ntotsios, E. Modelling, Simulation and Evaluation of Ground Vibration Caused by Rail Vehicles. Veh. Syst. Dyn. 2019, 57, 936–983. [Google Scholar] [CrossRef]
  2. Shen, Y.; Qi, X. Update on Diagnosis and Differential Diagnosis of Vestibular Migraine. Neurol. Sci. 2022, 43, 1659–1666. [Google Scholar] [CrossRef] [PubMed]
  3. Ugur, E.; Konukseven, B.O. The Potential Use of Virtual Reality in Vestibular Rehabilitation of Motion Sickness. Auris Nasus Larynx 2022, 49, 768–781. [Google Scholar] [CrossRef] [PubMed]
  4. Reason, J.T.; Brand, J.J. Motion Sickness; Academic Press: Cambridge, MA, USA, 1975. [Google Scholar]
  5. Oka, N.; Yoshino, K.; Yamamoto, K.; Takahashi, H.; Li, S.; Sugimachi, T.; Nakano, K.; Suda, Y.; Kato, T. Greater Activity in the Frontal Cortex on Left Curves: A Vector-Based FNIRS Study of Left and Right Curve Driving. PLoS ONE 2015, 10, e0127594. [Google Scholar] [CrossRef] [PubMed]
  6. Gavgani, A.M.; Wong, R.H.X.; Howe, P.R.C.; Hodgson, D.M.; Walker, F.R.; Nalivaiko, E. Cybersickness-Related Changes in Brain Hemodynamics: A Pilot Study Comparing Transcranial Doppler and near-Infrared Spectroscopy Assessments during a Virtual Ride on a Roller Coaster. Physiol. Behav. 2018, 191, 56–64. [Google Scholar] [CrossRef]
  7. Zhang, C.; Li, S.; Li, Y.; Li, S.E.; Nie, B. Analysis of Motion Sickness Associated Brain Activity Using FNIRS: A Driving Simulator Study. IEEE Access 2020, 8, 207415–207425. [Google Scholar] [CrossRef]
  8. Zhao, L.; Li, C.; Ji, L.; Yang, T. EEG Characteristics of Motion Sickness Subjects in Automatic Driving Mode Based on Virtual Reality Tests. J. Tsinghua Univ. (Sci. Technol.) 2020, 60, 993–998. [Google Scholar]
  9. Lim, H.K.; Ji, K.; Woo, Y.S.; Han, D.; Lee, D.-H.; Nam, S.G.; Jang, K.-M. Test-Retest Reliability of the Virtual Reality Sickness Evaluation Using Electroencephalography (EEG). Neurosci. Lett. 2020, 743, 135589. [Google Scholar] [CrossRef]
  10. Li, Z.; Zhao, L.; Chang, J.; Li, W.; Yang, M.; Li, C.; Wang, R.; Ji, L. EEG-Based Evaluation of Motion Sickness and Reducing Sensory Conflict in a Simulated Autonomous Driving Environment. In Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK, 11 July 2022; IEEE: New York, NY, USA, 2022; pp. 4026–4030. [Google Scholar]
  11. Hwang, J.-U.; Bang, J.-S.; Lee, S.-W. Classification of Motion Sickness Levels Using Multimodal Biosignals in Real Driving Conditions. In Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Prague, Czech Republic, 9 October 2022; IEEE: New York, NY, USA, 2022; pp. 1304–1309. [Google Scholar]
  12. Irmak, T.; Pool, D.M.; Happee, R. Objective and Subjective Responses to Motion Sickness: The Group and the Individual. Exp. Brain Res. 2021, 239, 515–531. [Google Scholar] [CrossRef]
  13. Tan, R.; Li, W.; Hu, F.; Xiao, X.; Li, S.; Xing, Y.; Wang, H.; Cao, D. Motion Sickness Detection for Intelligent Vehicles: A Wearable-Device-Based Approach. In Proceedings of the 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), Macao, China, 8 October 2022; IEEE: New York, NY, USA; pp. 4355–4362. [Google Scholar]
  14. Koch, A.; Cascorbi, I.; Westhofen, M.; Dafotakis, M.; Klapa, S.; Kuhtz-Buschbeck, J.P. The Neurophysiology and Treatment of Motion Sickness. Dtsch. Ärzteblatt Int. 2018, 115, 687. [Google Scholar] [CrossRef]
  15. Fuster, J.M. The Prefrontal Cortex Makes the Brain a Preadaptive System. Proc. IEEE 2014, 102, 417–426. [Google Scholar] [CrossRef]
  16. Acharya, D.; Mukherjea, A.; Cao, J.; Ruesch, A.; Schmitt, S.; Yang, J.; Smith, M.A.; Kainerstorfer, J.M. Non-Invasive Spectroscopy for Measuring Cerebral Tissue Oxygenation and Metabolism as a Function of Cerebral Perfusion Pressure. Metabolites 2022, 12, 667. [Google Scholar] [CrossRef] [PubMed]
  17. Boushel, R.; Langberg, H.; Olesen, J.; Gonzales-Alonzo, J.; Bülow, J.; Kjaer, M. Monitoring Tissue Oxygen Availability with near Infrared Spectroscopy (NIRS) in Health and Disease. Scand. J. Med. Sci. Sports 2001, 11, 213–222. [Google Scholar] [CrossRef]
  18. Chitnis, D.; Airantzis, D.; Highton, D.; Williams, R.; Phan, P.; Giagka, V.; Powell, S.; Cooper, R.J.; Tachtsidis, I.; Smith, M.; et al. Towards a Wearable near Infrared Spectroscopic Probe for Monitoring Concentrations of Multiple Chromophores in Biological Tissue in Vivo. Rev. Sci. Instrum. 2016, 87, 065112. [Google Scholar] [CrossRef] [PubMed]
  19. Eastmond, C.; Subedi, A.; De, S.; Intes, X. Deep Learning in FNIRS: A Review. Neurophotonics 2022, 9, 041411. [Google Scholar] [CrossRef] [PubMed]
  20. Rocco, G.; Lebrun, J.; Meste, O.; Magnie-Mauro, M.-N. A Chiral FNIRS Spotlight on Cerebellar Activation in a Finger Tapping Task. In Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Virtual Conference, 1 November 2021; IEEE: New York, NY, USA, 2021; pp. 1018–1021. [Google Scholar]
  21. Golding, J.F.; Gresty, M.A. Pathophysiology and Treatment of Motion Sickness. Curr. Opin. Neurol. 2015, 28, 83–88. [Google Scholar] [CrossRef]
  22. Ayaz, H.; Shewokis, P.A.; Bunce, S.C.; Onaral, B. Functional Near Infrared Spectroscopy Based Brain Computer Interface. U.S. Patent No. 9,946,344, 10 July 2018. [Google Scholar]
  23. Cao, N.; Gao, T. Noninvasive Tissue Blood Oxygenation Measurement Based on Near Infrared Spectroscopy (NIRS). In Proceedings of the 2009 3rd International Conference on Bioinformatics and Biomedical Engineering, Beijing, China, 11–13 June 2009; IEEE: New York, NY, USA, 2009; pp. 1–4. [Google Scholar]
  24. Molavi, B.; Dumont, G.A. Wavelet-Based Motion Artifact Removal for Functional near-Infrared Spectroscopy. Physiol. Meas. 2012, 33, 259–270. [Google Scholar] [CrossRef]
  25. Lin, C.-T.; King, J.-T.; Chuang, C.-H.; Ding, W.; Chuang, W.-Y.; Liao, L.-D.; Wang, Y.-K. Exploring the Brain Responses to Driving Fatigue Through Simultaneous EEG and FNIRS Measurements. Int. J. Neural Syst. 2020, 30, 1950018. [Google Scholar] [CrossRef]
  26. Zhou, X.; Sobczak, G.; McKay, C.M.; Litovsky, R.Y. Comparing FNIRS Signal Qualities between Approaches with and without Short Channels. PLoS ONE 2020, 15, e0244186. [Google Scholar] [CrossRef]
  27. Giller, C.A.; Hatab, M.R.; Giller, A.M. Giller Oscillations in Cerebral Blood Flow Detected with a Transcranial Doppler Index. J. Cereb. Blood Flow Metab. 1999, 19, 452–459. [Google Scholar] [CrossRef]
  28. Obrig, H.; Neufang, M.; Wenzel, R.; Kohl, M.; Steinbrink, J.; Einhäupl, K.; Villringer, A. Spontaneous Low Frequency Oscillations of Cerebral Hemodynamics and Metabolism in Human Adults. Neuroimage 2000, 12, 623–639. [Google Scholar] [CrossRef] [PubMed]
  29. Bertolini, G.; Straumann, D. Moving in a Moving World: A Review on Vestibular Motion Sickness. Front. Neurol. 2016, 7, 14. [Google Scholar] [CrossRef] [PubMed]
  30. Wang, X.; Ma, L.-C.; Shahdadian, S.; Wu, A.; Truong, N.C.D.; Liu, H. Metabolic Connectivity and Hemodynamic-Metabolic Coherence of Human Prefrontal Cortex at Rest and Post Photobiomodulation Assessed by Dual-Channel Broadband NIRS. Metabolites 2022, 12, 42. [Google Scholar] [CrossRef] [PubMed]
  31. Arie, R.; Brand, A.; Engelberg, S. Compressive Sensing and Sub-Nyquist Sampling. IEEE Instrum. Meas. Mag. 2020, 23, 94–101. [Google Scholar] [CrossRef]
  32. Kameyama, M.; Fukuda, M.; Yamagishi, Y.; Sato, T.; Uehara, T.; Ito, M.; Suto, T.; Mikuni, M. Frontal Lobe Function in Bipolar Disorder: A Multichannel near-Infrared Spectroscopy Study. Neuroimage 2006, 29, 172–184. [Google Scholar] [CrossRef]
  33. Huang, W.; Li, X.; Xie, H.; Qiao, T.; Zheng, Y.; Su, L.; Tang, Z.-M.; Dou, Z. Different Cortex Activation and Functional Connectivity in Executive Function Between Young and Elder People During Stroop Test: An FNIRS Study. Front. Aging Neurosci. 2022, 14, 864662. [Google Scholar] [CrossRef]
  34. Gao, T.; Zou, C.; Li, J.; Han, C.; Zhang, H.; Li, Y.; Tang, X.; Fan, Y. Identification of Moyamoya Disease Based on Cerebral Oxygen Saturation Signals Using Machine Learning Methods. J. Biophotonics 2022, 15, e202100388. [Google Scholar] [CrossRef]
  35. Ni, Y.; Sun, F.; Luo, Y.; Xiang, Z.; Sun, H. A Novel Heart Disease Classification Algorithm Based on Fourier Transform and Persistent Homology. In Proceedings of the 2022 IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA), Changchun, China, 25 February 2022; IEEE: New York, NY, USA, 2022; pp. 116–122. [Google Scholar]
  36. Li, M.; Liu, Y.; Zhi, S.; Wang, T.; Chu, F. Short-Time Fourier Transform Using Odd Symmetric Window Function. J. Dyn. Monit. Diagn. 2021, 1, 37–45. [Google Scholar] [CrossRef]
  37. Zhang, D.; Zhang, D. Wavelet Transform. In Fundamentals of Image Data Mining: Analysis, Features, Classification and Retrieval; Springer: Cham, Switzerland, 2019; pp. 35–44. [Google Scholar]
  38. Subasi, A.; Saikia, A.; Bagedo, K.; Singh, A.; Hazarika, A. EEG-Based Driver Fatigue Detection Using FAWT and Multiboosting Approaches. IEEE Trans. Ind. Inf. 2022, 18, 6602–6609. [Google Scholar] [CrossRef]
  39. Yan, H.; Xu, T.; Wang, P.; Zhang, L.; Hu, H.; Bai, Y. MEMS Hydrophone Signal Denoising and Baseline Drift Removal Algorithm Based on Parameter-Optimized Variational Mode Decomposition and Correlation Coefficient. Sensors 2019, 19, 4622. [Google Scholar] [CrossRef]
  40. Pooja; Pahuja, S.; Veer, K. Recent Approaches on Classification and Feature Extraction of EEG Signal: A Review. Robotica 2022, 40, 77–101. [Google Scholar] [CrossRef]
  41. Zhang, A.; Yang, B.; Huang, L. Feature Extraction of EEG Signals Using Power Spectral Entropy. In Proceedings of the 2008 International Conference on BioMedical Engineering and Informatics, Sanya, China, 28–30 May 2008; IEEE: New York, NY, USA, 2008; pp. 435–439. [Google Scholar]
  42. Choukèr, A.; Kaufmann, I.; Kreth, S.; Hauer, D.; Feuerecker, M.; Thieme, D.; Vogeser, M.; Thiel, M.; Schelling, G. Motion Sickness, Stress and the Endocannabinoid System. PLoS ONE 2010, 5, e10752. [Google Scholar] [CrossRef] [PubMed]
  43. Mareta, S.; Thenara, J.M.; Rivero, R.; Tan-Mullins, M. A Study of the Virtual Reality Cybersickness Impacts and Improvement Strategy towards the Overall Undergraduate Students’ Virtual Learning Experience. Interact. Technol. Smart Educ. 2022, 19, 460–481. [Google Scholar] [CrossRef]
  44. Dadgostar, M.; Setarehdan, S.K.; Shahzadi, S.; Akin, A. Classification of schizophrenia using SVM via fNIRS. Biomed. Eng. 2018, 30, 1850008. [Google Scholar] [CrossRef]
  45. Liu, S.; Wang, S.; Hu, C.; Qin, X.; Wang, J.; Kong, D. Development of a New NIR-Machine Learning Approach for Simultaneous Detection of Diesel Various Properties. Measurement 2022, 187, 110293. [Google Scholar] [CrossRef]
  46. Suthaharan, S. Support Vector Machine. In Machine Learning Models and Algorithms for Big Data Classification; Springer: Boston, MA, USA, 2016; pp. 207–235. [Google Scholar]
  47. Naji, M.A.; El Filali, S.; Aarika, K.; Benlahmar, E.H.; Abdelouhahid, R.A.; Debauche, O. Machine Learning Algorithms For Breast Cancer Prediction And Diagnosis. Procedia Comput. Sci. 2021, 191, 487–492. [Google Scholar] [CrossRef]
Figure 1. System architecture diagram.
Figure 1. System architecture diagram.
Diagnostics 13 01462 g001
Figure 2. fNIRS device and measurement principle with Subject.
Figure 2. fNIRS device and measurement principle with Subject.
Diagnostics 13 01462 g002
Figure 3. (ad) The fNIRS data preprocessing procedure, including the original data, Meanfilt data, MedFilt data, and MAS-corrected data. (e) Raw light-intensity signal, MeanFilt, MedFilt, and MAS.
Figure 3. (ad) The fNIRS data preprocessing procedure, including the original data, Meanfilt data, MedFilt data, and MAS-corrected data. (e) Raw light-intensity signal, MeanFilt, MedFilt, and MAS.
Diagnostics 13 01462 g003
Figure 4. The first component PC1 and the second principal component PC2 of Subject 1 in a set of experiments. The largest of them is marked with a red circle, and the channel corresponding to the red circle is selected for the specified channel.
Figure 4. The first component PC1 and the second principal component PC2 of Subject 1 in a set of experiments. The largest of them is marked with a red circle, and the channel corresponding to the red circle is selected for the specified channel.
Diagnostics 13 01462 g004
Figure 5. Relationship between wavelet layers and I–V band range.
Figure 5. Relationship between wavelet layers and I–V band range.
Diagnostics 13 01462 g005
Figure 6. Experiment flow chart with fNIRS signal recording.
Figure 6. Experiment flow chart with fNIRS signal recording.
Diagnostics 13 01462 g006
Figure 7. (a) is the test vehicle, (b) is the test route, and (c-1c-3) denotes one of the subjects.
Figure 7. (a) is the test vehicle, (b) is the test route, and (c-1c-3) denotes one of the subjects.
Diagnostics 13 01462 g007
Figure 8. (a) Classification of dizziness severity for Subject 1. (b) Classification of dizziness severity for Subjects 2–5. (c) Classification of dizziness severity for Subjects 6–9. (d) Classification of dizziness severity for Subjects 10–13.
Figure 8. (a) Classification of dizziness severity for Subject 1. (b) Classification of dizziness severity for Subjects 2–5. (c) Classification of dizziness severity for Subjects 6–9. (d) Classification of dizziness severity for Subjects 10–13.
Diagnostics 13 01462 g008aDiagnostics 13 01462 g008b
Figure 9. Confusion matrix plot of SVM output for all subjects’ motion sickness levels.
Figure 9. Confusion matrix plot of SVM output for all subjects’ motion sickness levels.
Diagnostics 13 01462 g009
Table 1. Band range of physiological signals in the five frequency bands.
Table 1. Band range of physiological signals in the five frequency bands.
Frequency BandFrequency (Hz)Physiological Meaning
I0.6–2.0Heart rate activity
II0.145–0.6Respiratory activity
III0.052–0.145Myogenic activity
IV0.021–0.052Neurogenic activity
V0.0095–0.021 Endothelial cell metabolic activity
Table 2. The -point motion sickness scale: Descriptions and corresponding scores for different degrees of motion sickness severity.
Table 2. The -point motion sickness scale: Descriptions and corresponding scores for different degrees of motion sickness severity.
ScoreDescription
0No motion sickness at all: participants have no sensation of motion sickness and feel normal.
1Slight motion sickness: participants have slight motion sickness but can still carry on with normal activities.
2Some motion sickness: participants experience motion sickness symptoms and may feel some discomfort but can still carry on with normal activities.
3Much motion sickness: participants experience moderate motion sickness symptoms such as dizziness and nausea, which affect their ability to carry on with normal activities.
4Extreme motion sickness: participants experience severe motion sickness symptoms such as severe dizziness and nausea, which prevent them from carrying on with normal activities.
5Unbearable motion sickness: participants experience extremely severe motion sickness symptoms such as unbearable dizziness, nausea, and vomiting, which require immediate cessation of the activity and medical attention.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ren, B.; Guan, W.; Zhou, Q. Study of Motion Sickness Model Based on fNIRS Multiband Features during Car Rides. Diagnostics 2023, 13, 1462. https://doi.org/10.3390/diagnostics13081462

AMA Style

Ren B, Guan W, Zhou Q. Study of Motion Sickness Model Based on fNIRS Multiband Features during Car Rides. Diagnostics. 2023; 13(8):1462. https://doi.org/10.3390/diagnostics13081462

Chicago/Turabian Style

Ren, Bin, Wanli Guan, and Qinyu Zhou. 2023. "Study of Motion Sickness Model Based on fNIRS Multiband Features during Car Rides" Diagnostics 13, no. 8: 1462. https://doi.org/10.3390/diagnostics13081462

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop