Towards Effective Emotion Detection: A Comprehensive Machine Learning Approach on EEG Signals
Abstract
:1. Introduction
1.1. Significance of Electroencephalography (EEG)
1.2. Machine Learning Applications for Emotion Detection from EEG Signals
2. Our Contribution
2.1. Gap Analysis
2.2. Research Questions
- Classifier Performance and Generalization: How do various machine learning classifiers perform in the task of emotion recognition using EEG data? Specifically, what is the accuracy of these classifiers in both training and validation scenarios, and how well do they generalize their predictions to new, unseen data instances?
- Relationship Between Model Complexity and Accuracy: Is there a relationship between the complexity of machine learning models and their accuracy in emotion recognition? Do more complex models consistently outperform simpler ones, or are there instances where simpler models achieve comparable accuracy?
- Inherent Model Advantages and Model Comparison: Are there certain machine learning classifiers that demonstrate inherent advantages in capturing the underlying patterns within EEG data for emotion recognition? How do these advantageous models compare to their counterparts in terms of accuracy and performance, and what insights can be gained from their differential performance?
2.3. Novelty of This Study
2.4. Significance of Our Work
3. Methodology
3.1. Dataset
3.2. Detailed Methodology
3.3. Classifiers
3.3.1. Random Forest
- Bootstrapping: Create multiple samples from the original dataset by using bootstrapping.
- Generation of Decision Trees: For each bootstrapped sample, build a decision tree where each tree continues to grow until all of its leaves contain only one class.
- Feature Selection: Select a random subset of features for each split in each tree.
- Classification: Classify new instances by using the majority vote of all the decision trees.
- Regression: Compute the average predictions of all the trees for regression problems.
- Pruning: Eliminate decision trees that do not have a significant impact on the final prediction to prevent overfitting.
- Repetition: Repeat steps 1–6 multiple times to form an ensemble of trees.
- Final Prediction: Make the final prediction based on the majority vote of all the trees in the ensemble.
3.3.2. Decision Tree
3.3.3. Logistic Regression
3.3.4. Support Vector Machine
3.3.5. Stochastic Gradient Descent
3.3.6. Nearest Centroid
- The centroid for each target class is computed during training.
- Say ’X’ at any point after training. Distances are calculated between the point X and the centroid of each class.
- From among all calculated distances, the shortest distance is chosen. The class is assigned to the centroid from which the given point is the shortest distance.
3.3.7. Naive Bayes
3.4. Evaluation Metrics
3.5. Experimental Settings
4. Results
- For random forest, across training data proportions of 90%, 80%, 70%, and 60%, the accuracy was consistently 1.
- Similarly, decision tree exhibited identical results across all training data proportions.
- Logistic regression attained an accuracy of 1 across all training datasets.
- SVM achieved an accuracy of 0.98 for training data proportions of 90% and 80%, 0.97 for 60%, and 0.98 for 70%.
- SGD demonstrated an accuracy of 0.99 for 90%, 1 for 80% and 70%, and 0.99 for the 60% training data.
- Nearest centroid yielded an accuracy of 0.78 for 90%, 0.8 for 80% and 70%, and 0.79 for the 60% training data.
- Gaussian naive Bayes (GNB) showed an accuracy of 0.65 across all training data proportions.
- Bernoulli naive Bayes (BNB) achieved an accuracy of 0.83 for 90%, 0.84 for 80% and 70%, and 0.83 for the 60% training data.
- For random forest, the accuracy was 0.99 for the 10% validation data, and 0.98 for 20%, 30%, and 40% validation data.
- Decision tree demonstrated an accuracy of 0.97 for the 10% validation data, 0.95 for 20% validation data, 0.96 for 30% validation data, and 0.94 for 40% validation data.
- Logistic regression exhibited an accuracy of 0.97 for 10% validation data, and 0.96 for 20%, 30%, and 40% validation data.
- SVM achieved a consistent accuracy of 0.93 across all validation datasets.
- SGD’s accuracy was 0.96 for the 10% validation data, 0.94 for 20% validation data, 0.95 for 30% validation data, and 0.93 for 40% validation data.
- Nearest centroid yielded an accuracy of 0.7 for the 10% validation data, 0.77 for 20% validation data, 0.78 for 30% validation data, and 0.77 for 40% validation data.
- Gaussian naive Bayes displayed an accuracy of 0.63 for the 10% validation data, and 0.65 for 20%, 30%, and 40% validation data.
- Bernoulli naive Bayes attained an accuracy of 0.81 for the 10% validation data, 0.82 for 20% testing data, 0.83 for 30% validation data, and 0.84 for 40% validation data.
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
EEG | electroencephalogram |
SVM | Support Vector Machine |
SGD | Stochastic Gradient Descent |
OVA | one versus all |
kNN | k-Nearest Neighbors |
BNB | Bernoulli Naive Bayes |
GNB | Gaussian Naive Bayes |
References
- Gavrilova, M.L.; Anzum, F.; Hossain Bari, A.; Bhatia, Y.; Iffath, F.; Ohi, Q.; Shopon, M.; Wahid, Z. A multifaceted role of biometrics in online security, privacy, and trustworthy decision making. In Breakthroughs in Digital Biometrics and Forensics; Springer: Berlin/Heidelberg, Germany, 2022; pp. 303–324. [Google Scholar]
- Sharma, A.; Sharma, K.; Kumar, A. Real-time emotional health detection using fine-tuned transfer networks with multimodal fusion. Neural Comput. Appl. 2022, 35, 22935–22948. [Google Scholar] [CrossRef]
- Segal, A.; Kaniušonytė, G.; Bakaitytė, A.; Žukauskienė, R.; Santtila, P. The Effects of Emotions on the Assessment of Child Sexual Abuse Interviews. J. Police Crim. Psychol. 2023, 38, 826–837. [Google Scholar] [CrossRef]
- Bhatt, P.; Sethi, A.; Tasgaonkar, V.; Shroff, J.; Pendharkar, I.; Desai, A.; Sinha, P.; Deshpande, A.; Joshi, G.; Rahate, A.; et al. Machine learning for cognitive behavioral analysis: Datasets, methods, paradigms, and research directions. Brain Inform. 2023, 10, 18. [Google Scholar] [CrossRef]
- Tesfagergish, S.G.; Kapočiūtė-Dzikienė, J.; Damaševičius, R. Zero-shot emotion detection for semi-supervised sentiment analysis using sentence transformers and ensemble learning. Appl. Sci. 2022, 12, 8662. [Google Scholar] [CrossRef]
- Liang, N.Y.; Saratchandran, P.; Huang, G.B.; Sundararajan, N. Classification of mental tasks from eeg signals using extreme learning machine. Int. J. Neural Syst. 2006, 16, 29–38. [Google Scholar] [CrossRef] [PubMed]
- Vempati, R.; Sharma, L.D. A Systematic Review on Automated Human Emotion Recognition using Electroencephalogram Signals and Artificial Intelligence. Results Eng. 2023, 18, 101027. [Google Scholar] [CrossRef]
- Dalilian, F.; Nembhard, D. Biometrically Measured Affect for Screen-Based Drone Pilot Skill Acquisition. Int. J. Hum.-Comput. Interact. 2023, 1–16. [Google Scholar] [CrossRef]
- Merlin Praveena, D.; Angelin Sarah, D.; Thomas George, S. Deep learning techniques for EEG signal applications—A review. IETE J. Res. 2022, 68, 3030–3037. [Google Scholar] [CrossRef]
- Delisle-Rodriguez, D.; de Oliveira Junior, H.L.; da Silva, J.C.; de Souza, M.L.; Bastos, T.; Nakamura-Palacios, E.M.; Frizera-Neto, A. Multi-channel EEG-based BCI using regression and classification methods for attention training by serious game. Biomed. Signal Process. Control 2023, 85, 104937. [Google Scholar] [CrossRef]
- Taheri Gorji, H.; Wilson, N.; VanBree, J.; Hoffmann, B.; Petros, T.; Tavakolian, K. Using machine learning methods and EEG to discriminate aircraft pilot cognitive workload during flight. Sci. Rep. 2023, 13, 2507. [Google Scholar] [CrossRef]
- Adochiei, I.R.; Stirbu, O.; Nicolescu, S.T.; Paraschiv, R.V.; Bănică, C.K.; Adochiei, F.C. Human Behavior, Recognition, and Interpretation System using Physiological Signals. In Proceedings of the 2023 13th International Symposium on Advanced Topics in Electrical Engineering (ATEE), Bucharest, Romania, 23–25 March 2023; pp. 1–4. [Google Scholar]
- Dadebayev, D.; Goh, W.W.; Tan, E.X. EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques. J. King Saud Univ.-Comput. Inf. Sci. 2022, 34, 4385–4401. [Google Scholar] [CrossRef]
- Mikhail, M.; Ayat, K.E.; Coan, J.A.; Allen, J.J. Using minimal number of electrodes for emotion detection using brain signals produced from a new elicitation technique. Int. J. Auton. Adapt. Commun. Syst. 2013, 6, 80. [Google Scholar] [CrossRef]
- Simpraga, S.; Alvarez-Jimenez, R.; Mansvelder, H.D.; van Gerven, J.M.A.; Groeneveld, G.J.; Poil, S.S.; Linkenkaer-Hansen, K. EEG machine learning for accurate detection of cholinergic intervention and Alzheimer’s disease. Sci. Rep. 2017, 7, 5775. [Google Scholar] [CrossRef] [PubMed]
- Rasheed, K.; Qayyum, A.; Qadir, J.; Sivathamboo, S.; Kwan, P.; Kuhlmann, L.; O’Brien, T.; Razi, A. Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review. IEEE Rev. Biomed. Eng. 2021, 14, 139–155. [Google Scholar] [CrossRef]
- Hosseini, M.P.; Hosseini, A.; Ahi, K. A Review on Machine Learning for EEG Signal Processing in Bioengineering. IEEE Rev. Biomed. Eng. 2021, 14, 204–218. [Google Scholar] [CrossRef]
- Sayeed, M.A.; Mohanty, S.P.; Kougianos, E.; Zaveri, H.P. Neuro-Detect: A Machine Learning-Based Fast and Accurate Seizure Detection System in the IoMT. IEEE Trans. Consum. Electron. 2019, 65, 359–368. [Google Scholar] [CrossRef]
- Jamil, N.; Belkacem, A.N.; Ouhbi, S.; Lakas, A. Noninvasive electroencephalography equipment for assistive, adaptive, and rehabilitative brain–computer interfaces: A systematic literature review. Sensors 2021, 21, 4754. [Google Scholar] [CrossRef]
- Hosseinifard, B.; Moradi, M.H.; Rostami, R. Classifying depression patients and normal subjects using machine learning techniques and nonlinear features from EEG signal. Comput. Methods Programs Biomed. 2013, 109, 339–345. [Google Scholar] [CrossRef]
- Yuan, Q.; Zhou, W.; Li, S.; Cai, D. Epileptic EEG classification based on extreme learning machine and nonlinear features. Epilepsy Res. 2011, 96, 29–38. [Google Scholar] [CrossRef]
- Kamble, K.; Sengupta, J. A comprehensive survey on emotion recognition based on electroencephalograph (EEG) signals. Multimed. Tools Appl. 2023, 82, 1–36. [Google Scholar] [CrossRef]
- Li, G.; Ouyang, D.; Yuan, Y.; Li, W.; Guo, Z.; Qu, X.; Green, P. An EEG data processing approach for emotion recognition. IEEE Sens. J. 2022, 22, 10751–10763. [Google Scholar] [CrossRef]
- Alhalaseh, R.; Alasasfeh, S. Machine-learning-based emotion recognition system using EEG signals. Computers 2020, 9, 95. [Google Scholar] [CrossRef]
- Liu, H.; Zhang, Y.; Li, Y.; Kong, X. Review on emotion recognition based on electroencephalography. Front. Comput. Neurosci. 2021, 15, 84. [Google Scholar]
- Houssein, E.H.; Hammad, A.; Ali, A.A. Human emotion recognition from EEG-based brain–computer interface using machine learning: A comprehensive review. Neural Comput. Appl. 2022, 34, 12527–12557. [Google Scholar] [CrossRef]
- Koelstra, S.; Muhl, C.; Soleymani, M.; Lee, J.S.; Yazdani, A.; Ebrahimi, T.; Pun, T.; Nijholt, A.; Patras, I. Deap: A database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 2011, 3, 18–31. [Google Scholar] [CrossRef]
- Duan, R.N.; Zhu, J.Y.; Lu, B.L. Differential entropy feature for EEG-based emotion classification. In Proceedings of the 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego, CA, USA, 6–8 November 2013; pp. 81–84. [Google Scholar]
- Bird, J.J.; Ekart, A.; Buckingham, C.D.; Faria, D.R. Mental emotional sentiment classification with an eeg-based brain-machine interface. In Proceedings of the International Conference on Digital Image and Signal Processing (DISP’19), Oxford, UK, 23–30 April 2019. [Google Scholar]
- Bird, J.J. EEG Brainwave Dataset: Feeling Emotions. 2018. Available online: https://www.kaggle.com/datasets/birdy654/eeg-brainwave-dataset-feeling-emotions (accessed on 10 April 2023).
- Katsigiannis, S.; Ramzan, N. DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE J. Biomed. Health Inform. 2018, 22, 98–107. [Google Scholar] [CrossRef]
- Zhang, L. Seed Dataset: A Dataset Collection for Various Purposes Using EEG Signals. 2013. Available online: https://bcmi.sjtu.edu.cn/home/seed/index.html (accessed on 25 August 2023).
- Bhuvaneswari, P.; Satheesh Kumar, J. Support Vector Machine Technique for EEG Signals. Int. J. Comput. Appl. 2013, 63, 1–5. [Google Scholar] [CrossRef]
- Wu, W.; Nagarajan, S.; Chen, Z. Bayesian Machine Learning: EEG/MEG signal processing measurements. IEEE Signal Process. Mag. 2016, 33, 14–36. [Google Scholar] [CrossRef]
- Al Zoubi, O.; Ki Wong, C.; Kuplicki, R.T.; Yeh, H.w.; Mayeli, A.; Refai, H.; Paulus, M.; Bodurka, J. Predicting Age From Brain EEG Signals—A Machine Learning Approach. Front. Aging Neurosci. 2018, 10, 184. [Google Scholar] [CrossRef]
- Ding, S.; Zhang, N.; Xu, X.; Guo, L.; Zhang, J. Deep Extreme Learning Machine and Its Application in EEG Classification. Math. Probl. Eng. 2015, 2015, 129021. [Google Scholar] [CrossRef]
Datasets | Reference | Year of Publication | # of Classes | # of People (Male + Female) | # of Minutes | Available From |
---|---|---|---|---|---|---|
EEG Brainwave Dataset: Feeling Emotions | Bird et al. [29] | 2019 | 3 (Negative, Positive, & Neutral) | 2 (1 + 1) | 9 | Kaggle ([30]) |
DREAMER | Katsigiannis et al. [31] | 2017 | 3 (Violence, Arousal & Dominance) | 23 (14 + 9) | 59.7 | Zenodo ([31]) |
SEED_EEG | Duan et al. [28] | 2013 | 3 (Negative, Positive, & Neutral) | 15 (7 + 8) | 960 | BCMI ([32]) |
DEAP | Koelstra et al. [27] | 2011 | 3 (Arousal, Violence, & Like/Dislike) | 32 | 1280 | Prof. Ioannis Patras ([27]) |
Classifiers | 90 Percent Data for Training | 80 Percent Data for Training | 70 Percent Data for Training | 60 Percent Data For Training |
---|---|---|---|---|
Random Forest | 1 | 1 | 1 | 1 |
Decision Tree | 1 | 1 | 1 | 1 |
Logistic Regression | 1 | 1 | 1 | 1 |
SVM | 0.98 | 0.98 | 0.98 | 0.97 |
SGD | 0.99 | 1 | 1 | 0.99 |
Nearest Centroid | 0.78 | 0.8 | 0.8 | 0.79 |
Gaussian NB | 0.65 | 0.65 | 0.65 | 0.65 |
Bernoulli NB | 0.83 | 0.84 | 0.84 | 0.83 |
Classifiers | 10 Percent Data for Validation | 20 Percent Data for Validation | 30 Percent Data for Validation | 40 Percent Data for Validation |
---|---|---|---|---|
Random Forest | 0.99 | 0.98 | 0.98 | 0.98 |
Decision Tree | 0.97 | 0.95 | 0.96 | 0.94 |
Logistic Regression | 0.97 | 0.96 | 0.96 | 0.96 |
SVM | 0.93 | 0.94 | 0.94 | 0.94 |
SGD | 0.96 | 0.94 | 0.95 | 0.93 |
Nearest Centroid | 0.7 | 0.77 | 0.78 | 0.77 |
Gaussian NB | 0.63 | 0.65 | 0.66 | 0.65 |
Bernoulli NB | 0.81 | 0.82 | 0.83 | 0.84 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ul Hassan, I.; Ali, R.H.; Abideen, Z.u.; Ijaz, A.Z.; Khan, T.A. Towards Effective Emotion Detection: A Comprehensive Machine Learning Approach on EEG Signals. BioMedInformatics 2023, 3, 1083-1100. https://doi.org/10.3390/biomedinformatics3040065
Ul Hassan I, Ali RH, Abideen Zu, Ijaz AZ, Khan TA. Towards Effective Emotion Detection: A Comprehensive Machine Learning Approach on EEG Signals. BioMedInformatics. 2023; 3(4):1083-1100. https://doi.org/10.3390/biomedinformatics3040065
Chicago/Turabian StyleUl Hassan, Ietezaz, Raja Hashim Ali, Zain ul Abideen, Ali Zeeshan Ijaz, and Talha Ali Khan. 2023. "Towards Effective Emotion Detection: A Comprehensive Machine Learning Approach on EEG Signals" BioMedInformatics 3, no. 4: 1083-1100. https://doi.org/10.3390/biomedinformatics3040065
APA StyleUl Hassan, I., Ali, R. H., Abideen, Z. u., Ijaz, A. Z., & Khan, T. A. (2023). Towards Effective Emotion Detection: A Comprehensive Machine Learning Approach on EEG Signals. BioMedInformatics, 3(4), 1083-1100. https://doi.org/10.3390/biomedinformatics3040065