EEG-Based Emotion Dynamics Recognition Using Hybrid AI Models for Cybersecurity
Abstract
1. Introduction
1.1. Emotions in Computer Vision Systems
1.2. Emotions and Cognitive States from EEG and Other Biosignals
- Preprocessing (bandpass filtering, artifact removal, re-referencing);
- Feature extraction in the θ (4–8 Hz), α (8–14 Hz), β (14–31 Hz), and γ bands;
- Subsequent multi-class classification of emotional states and a neutral baseline.
1.3. Multimodal Architectures: FER and EEG Integration
1.4. Neurophysiological Susceptibility to Phishing and Neurophishing
- Increased beta activity (14–31 Hz): associated with active cognitive strain and decision-making under pressure [21];
- Decreased alpha activity (8–14 Hz): reflects a decrease in the relaxed state and activation of the frontal cortex [22];
- Changes in the P300 component (event-related potential): a decrease in amplitude is recorded under high cognitive load and among individuals more susceptible to deception, making this indicator a candidate for online vulnerability assessment [42].
1.5. Integration of Emotions, EEG, and Anti-Phishing Protection
2. Theoretical Part
2.1. Description of the Subject Area
2.2. Wavelet Transform
2.3. Stockwell Transformation
2.4. Wavelet-Based KAN
2.5. Wavelet-KAN Approximation
Neurophysiological Rationale for KAN in EEG Processing
3. Materials and Methods
3.1. Datasets and Experimental Protocol
- Clips ranging from 2 to 4 min in length were selected, each evoking one of five emotions: happiness, sadness, fear, disgust, and neutral;
- EEG recording devices were installed on 16 participants aged 19 to 24 years (including 10 women and 6 men), and they viewed three randomly selected clips (each clip was used three times);
- The obtained results were saved as .cnt files, which were subsequently used for analysis.
- Bandpass filtering: EEG signals are bandpass-filtered in the conventional 1–50 Hz range to remove DC drift and high-frequency noise (including power line interference), consistent with prior EEG emotion recognition studies.
- Artifact handling: For SEED-V, standard artifact handling is applied, including removal of obviously corrupted segments and high-amplitude artifacts. For the Mendeley dataset (used only at inference), minimal preprocessing is applied due to the lower channel count and the auxiliary role of this dataset.
- Segmentation: Continuous recordings are segmented into fixed-length windows in the range of 1–4 s, consistent with the quasi-stationarity assumption required for the Kolmogorov–Arnold framework and the window-based training of WS-KAN-EEGNet.
- z-score normalization: Per-channel z-score normalization is applied within each window to constrain inputs to a bounded subset of , which is consistent with the compactness assumptions used in the KAT-based analysis.
- Time–frequency transform: A continuous wavelet transform (Morlet, cmor2.0-1.0) or S-transform is applied to each channel window to obtain time–frequency representations for subsequent processing.
- Sobel filtering: Edge enhancement is applied to the resulting time–frequency maps (Sobel filtering) for the 2D branch to emphasize salient spectral–temporal structures.
3.2. Data Processing Tools
3.3. Proposed Data Processing Algorithm
3.3.1. Model Testing Steps
- EEG data are recorded and labeled by EEG specialists; in this work, we use pre-labeled recordings from the SEED-V and TSST-based Mendeley datasets [58].
- The obtained data undergo a preprocessing step, which includes application of the continuous wavelet transform or Stockwell transform to the time series, followed by Sobel filtering of the resulting time–frequency representations and z-score normalization.
- Training and test datasets are formed according to a subject-independent 5-fold cross-validation protocol on SEED-V, where in each fold the data from 12–13 subjects are used for training and the data from the remaining 3–4 subjects are used for testing, with no overlap of subjects between the sets.
- The 1D-EEG WKAN and 2D-ST-KANCNN branches are trained on the SEED-V training folds using the protocol described in Section 3.3.3; after convergence, their parameters are frozen, and the ensemble fusion module with two learnable scalar weights is trained on SEED-V for an additional number of epochs.
- The trained WS-KAN-EEGNet model is evaluated on the SEED-V test folds to obtain classification metrics and, without any fine-tuning, is then applied to the TSST stress recordings from the Mendeley dataset to produce temporal trajectories of emotion probabilities, which are further analyzed in the context of stress and phishing vulnerability.
3.3.2. Preprocessing
3.3.3. Model Construction
Model Working Directly with EEG Graphs
Model for Preprocessed 2D EEG Images
Ensemble Model, WS-KAN-EEGNet
Kolmogorov–Arnold Layers in the Model Architecture





4. Experiments and Results
4.1. Overall Performance on SEED-V
4.2. Temporal Emotion Dynamics and Cross-Dataset Application
4.3. Feature Complementarity Analysis
4.4. KAN Function Interpretability Analysis
4.5. Wavelet Selection Sensitivity Analysis
4.6. Confusion Matrix and Error Cost Analysis
5. Discussion
5.1. Limitations
5.2. Future Directions
- Dedicated phishing-EEG dataset. Design and collect an EEG dataset with realistic phishing/smishing stimuli (email/SMS simulations with varying urgency, authority, and loss-aversion cues), including simultaneous eye-tracking and galvanic skin response recordings, behavioral metrics (response time, click-through rate), and binary vulnerability labels.
- Semantic analysis integration. Combine the EEG emotion classifier with NLP models analyzing phishing message text. Specifically, extract urgency cues, authority markers, and loss-framing features from message content using transformer-based text classifiers, and correlate these textual features with the corresponding EEG emotional response, creating a joint neurolinguistic vulnerability model.
- Topology-aware KAN integration. Combine KAN branches with graph neural networks (e.g., STGATE-like architectures [18]) to explicitly model electrode spatial connectivity, potentially improving cross-subject generalization through topology-aware representations.
- Real-time edge deployment. Develop a prototype system on portable BCI devices with edge-computing inference, including an adaptive interface warning system that triggers alerts when the fear probability trajectory exceeds a calibrated threshold.
- Adversarial robustness. Conduct systematic evaluation using FGSM, PGD, and C&W attacks adapted for time-series EEG, and develop defense mechanisms (adversarial training, certified robustness bounds) for EEG classifiers in security applications.
- KAN-SSM temporal modeling. Replace the current per-window approach with continuous KAN-based state-space models [57] for capturing smooth emotional transitions without the quasi-stationarity assumption.
- Expanded demographics. Increase sample size to 50+ subjects with diverse age groups (18–65), cultural backgrounds, and neurological profiles to improve demographic generalizability.
5.3. TSST-to-Phishing Cognitive Mapping
5.4. Scalability and Computational Cost
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
| Abbreviation | Definition |
| BCI | Brain–Computer Interface |
| CNN | Convolutional Neural Network |
| CWT | Continuous Wavelet Transform |
| DWT | Discrete Wavelet Transform |
| EDA | Electrodermal Activity |
| EEG | Electroencephalography |
| FC | Fully Connected |
| FER | Facial Expression Recognition |
| HPA | Hypothalamic–Pituitary–Adrenal |
| HRV | Heart Rate Variability |
| KAN | Kolmogorov–Arnold Network |
| LSTM | Long Short-Term Memory |
| MLP | Multilayer Perceptron |
| SE | Squeeze-and-Excitation |
| STFT | Short-Time Fourier Transform |
| TSST | Trier Social Stress Test |
| t-SNE | t-Distributed Stochastic Neighbor Embedding |
| WKAN | Wavelet-KAN |
References
- Pleshakova, E.; Osipov, A.; Gataullin, S.; Gataullin, T.; Vasilakos, A. Next Gen Cybersecurity Paradigm Towards Artificial General Intelligence: Russian Market Challenges and Future Global Technological Trends. J. Comput. Virol. Hacking Tech. 2024, 20, 429–440. [Google Scholar] [CrossRef] [Scilit]
- Yenduri, G.; Ramalingam, M.; Selvi, G.C.; Supriya, Y.; Srivastava, G.; Maddikunta, P.K.R.; Raj, G.D.; Jhaveri, R.H.; Prabadevi, B.; Wang, W.; et al. GPT (Generative Pre-Trained Transformer)—A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions. IEEE Access 2024, 12, 54608–54649. [Google Scholar] [CrossRef] [Scilit]
- Ivanyuk, V. Forecasting of Digital Financial Crimes in Russia Based on Machine Learning Methods. J. Comput. Virol. Hacking Tech. 2024, 20, 349–362. [Google Scholar] [CrossRef] [Scilit]
- Andriyanov, N.A.; Dementiev, V.E. Optimization of Face Recognition Systems for Implementation in Embedded Systems. Pattern Recognit. Image Anal. 2024, 34, 1245–1254. [Google Scholar] [CrossRef] [Scilit]
- Boltachev, E. Potential Cyber Threats of Adversarial Attacks on Autonomous Driving Models. J. Comput. Virol. Hacking Tech. 2024, 20, 363–373. [Google Scholar] [CrossRef] [Scilit]
- Bespalova, N.; Bylevsky, P. Source Code Obfuscation Assessment Techniques for Remote Financial Services. In 2024 17th International Conference on Management of Large-Scale System Development (MLSD); IEEE: New York, NY, USA, 2024. [Google Scholar]
- Chechkin, A.; Pleshakova, E.; Gataullin, S. A Hybrid KAN-BiLSTM Transformer with Multi-Domain Dynamic Attention Model for Cybersecurity. Technologies 2025, 13, 223. [Google Scholar] [CrossRef] [Scilit]
- Osipov, A.; Pleshakova, E.; Liu, Y.; Gataullin, S. Machine Learning Methods for Speech Emotion Recognition on Telecommunication Systems. J. Comput. Virol. Hacking Tech. 2024, 20, 415–428. [Google Scholar] [CrossRef] [Scilit]
- Osipov, A.V.; Sapozhnikov, A.E.; Pleshakova, E.S.; Gataullin, S.T. Machine Learning Methods for Recognizing the Emotional State of a Telecommunications System Sub-scriber. J. Inf. Technol. Comput. Syst. 2024, 1, 23–35. [Google Scholar]
- Sakovich, N.; Aksenov, D.; Pleshakova, E.; Gataullin, S. Wavelet-Based Optimization and Numerical Computing for Fault Detection Method—Signal Fault Locali-zation and Classification Algorithm. Algorithms 2025, 18, 217. [Google Scholar] [CrossRef] [Scilit]
- Beketov, S.M.; Zubkova, D.A.; Gintciak, A.M.; Burlutskaya, Z.V.; Redko, S.G. Modern Optimization Methods and Their Application Features. Russ. Technol. J. 2025, 13, 78–94. [Google Scholar] [CrossRef] [Scilit]
- Ullah, S.; Ou, J.; Xie, Y.; Tian, W. Facial Expression Recognition (FER) Survey: A Vision, Architectural Elements, and Future Directions. PeerJ Comput. Sci. 2024, 10, e2024. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Shahid, A. A Survey on Facial Expression Recognition: Modality, Methodologies, Challenges and Emerging Topics. TechRxiv 2023. [Google Scholar] [CrossRef] [Scilit]
- Alkan, N. Recognition and Misclassification Patterns of Basic Emotional Facial Expressions: An Eye-Tracking Study in Young Healthy Adults. J. Eye Mov. Res. 2025, 18, 53. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Lawhern, V.J.; Solon, A.J.; Waytowich, N.R.; Gordon, S.M.; Hung, C.P.; Lance, B.J. EEGNet: A Compact Convolutional Neural Network for EEG-Based Brain–Computer Interfaces. J. Neural Eng. 2018, 15, 056013. [Google Scholar] [CrossRef] [Scilit]
- Schirrmeister, R.T.; Springenberg, J.T.; Fiederer, L.D.J.; Glasstetter, M.; Eggensperger, K.; Tangermann, M.; Hutter, F.; Burgard, W.; Ball, T. Deep Learning with Convolutional Neural Networks for EEG Decoding and Visualization. Hum. Brain Mapp. 2017, 38, 5391–5420. [Google Scholar] [CrossRef] [Scilit]
- Zhang, Z.; Zhong, S.H.; Liu, Y. TorchEEGEMO: A deep learning toolbox towards EEG-based emotion recognition. Expert Syst. Appl. 2024, 249, 123550. [Google Scholar] [CrossRef] [Scilit]
- Li, J.; Pan, W.; Huang, H.; Pan, J.; Wang, F. STGATE: Spatial-Temporal Graph Attention Network with a Transformer Encoder for EEG-Based Emotion Recognition. Front. Hum. Neurosci. 2023, 17, 1169949. [Google Scholar] [CrossRef] [Scilit]
- Wu, X.; Ju, X.; Dai, S.; Li, X.; Li, M. Multi-Source Domain Adaptation for EEG Emotion Recognition Based on Inter-Domain Sample Hybridization. Front. Hum. Neurosci. 2024, 18, 1464431. [Google Scholar] [CrossRef] [Scilit]
- Xiao, Y.; Zhang, Y.; Peng, X.; Han, S.; Zheng, X.; Fang, D.; Chen, X. Multi-Source EEG Emotion Recognition via Dynamic Contrastive Domain Adaptation. Biomed. Signal Process. Control 2025, 102, 107337. [Google Scholar] [CrossRef] [Scilit]
- Ullah, S.; Ou, J.; Xie, Y.; Tian, W. Wearable EEG-Based Brain–Computer Interface for Stress Monitoring. NeuroSci 2024, 5, 407–428. [Google Scholar] [CrossRef] [Scilit]
- Mai, N.-D.; Chung, W.-Y. On-Chip Mental Stress Detection: Integrating a Wearable Behind-the-Ear EEG Device with Embed-ded Tiny Neural Network. IEEE J. Biomed. Health Inform. 2025, 29, 1872–1885. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Montañez, R.; Golob, E.; Xu, S. Human Cognition Through the Lens of Social Engineering Cyberattacks. Front. Psychol. 2020, 11, 1755. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Yao, Y.; Zheng, K.; Wu, B.; Wu, C.; Gao, J.; Wang, J.; Yang, M. The Psychological Manipulation of Phishing Emails: A Cognitive Bias Approach. Comput. Mater. Contin. 2025, 85, 4753–4776. [Google Scholar] [CrossRef] [Scilit]
- Klütsch, J.; Schwab, J.; Böffel, C.; Zimmermann, V.; Schlittmeier, S.J. Friend or Phisher: How Known Senders and Fear of Missing Out Affect Young Adults’ Phishing Susceptibility on Social Media. Humanit. Soc. Sci. Commun. 2024, 11, 1145. [Google Scholar] [CrossRef] [Scilit]
- Huang, Z.-Y.; Chiang, C.-C.; Chen, J.-H.; Chen, Y.-C.; Chung, H.-L.; Cai, Y.-P.; Hsu, H.-C. A Study on Computer Vision for Facial Emotion Recognition. Sci. Rep. 2023, 13, 35446. [Google Scholar] [CrossRef] [Scilit]
- Mo, F.; Gu, J.; Zhao, K.; Fu, X. Confusion Effects of Facial Expression Recognition in Patients with Major Depressive Disorder and Healthy Controls. Front. Psychol. 2021, 12, 703888. [Google Scholar] [CrossRef] [Scilit]
- Lee, S.C.; Lin, G.H.; Shih, C.L.; Chen, K.W.; Liu, C.C.; Kuo, C.J.; Hsieh, C.L. Error Patterns of Facial Emotion Recognition in Patients with Schizophrenia. J. Affect. Disord. 2022, 300, 441–448. [Google Scholar] [CrossRef] [Scilit]
- Martínez-González, A.E.; Veas, A. Identification of Emotions and Physiological Response in Individuals with Moderate Intellectual Disability. Int. J. Dev. Disabil. 2021, 67, 406–411. [Google Scholar] [CrossRef] [Scilit]
- Aly, M.; Alotaibi, N.S. A Comprehensive Deep Learning Framework for Real-Time Emotion Detection in Online Learning Using Hybrid Models. Sci. Rep. 2025, 15, 42012. [Google Scholar] [CrossRef] [Scilit]
- Kumar, G.S.; Cheriyan, J.; Aparna, N.; Swathy, J. Unleashing Facial Expression Recognition for Stress Detection Using Deep CNN Model. Procedia Comput. Sci. 2025, 259, 306–315. [Google Scholar] [CrossRef] [Scilit]
- Li, T.-H.; Liu, W.; Zheng, W.-L.; Lu, B.-L. Classification of Five Emotions from EEG and Eye Movement Signals: Discrimination Ability and Stability over Time. In Proceedings of the 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), San Francisco, CA, USA, 20–23 March 2019; pp. 607–610. [Google Scholar] [CrossRef] [Scilit]
- Kumar, G.S.; Sampathila, N.; Martis, R.J. Classification of Human Emotional States Based on Valence-Arousal Scale Using Electroencephalogram. J. Med. Signals Sens. 2023, 13, 173–182. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Fiorini, L.; Bossi, F.; Di Gruttola, F. EEG-Based Emotional Valence and Emotion Regulation Classification: A Data-Centric and Explainable Approach. Sci. Rep. 2024, 14, 24046. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ang, K.K.; Chin, Z.Y.; Wang, C.; Guan, C.; Zhang, H. Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b. Front. Neuro-Sci. 2012, 6, 39. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Du, X.; Ma, C.; Zhang, G.; Li, J.; Lai, Y.-K.; Zhao, G.; Deng, X.; Liu, Y.-J.; Wang, H. An Efficient LSTM Network for Emotion Recognition from Multichannel EEG Signals. IEEE Trans. Affect. Comput. 2022, 13, 1528–1540. [Google Scholar] [CrossRef] [Scilit]
- Pan, J.; Fang, W.; Zhang, Z.; Chen, B.; Zhang, Z.; Wang, S. Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG. IEEE Open J. Eng. Med. Biol. 2023, 5, 396–403. [Google Scholar] [CrossRef] [Scilit]
- Liu, R.; Chao, Y.; Ma, X.; Sha, X.; Sun, L.; Li, S.; Chang, S. ERTNet: An Interpretable Transformer-Based Framework for EEG Emotion Recognition. Front. Neurosci. 2024, 18, 1320645. [Google Scholar] [CrossRef] [Scilit]
- Hazmoune, S.; Bougamouza, F. Using Transformers for Multimodal Emotion Recognition: Taxonomies and State of the Art Review. Eng. Appl. Artif. Intell. 2024, 133, 108339. [Google Scholar] [CrossRef] [Scilit]
- Bilotti, U.; Bisogni, C.; De Marsico, M.; Tramonte, S. Multimodal Emotion Recognition via Convolutional Neural Networks: Comparison of Different Strategies on Two Multimodal Datasets. Eng. Appl. Artif. Intell. 2024, 130, 107708. [Google Scholar] [CrossRef] [Scilit]
- Nasser, G.; Morrison, B.W.; Bayl-Smith, P.; Taib, R.; Gayed, M.; Wiggins, M.W. The Role of Cue Utilization and Cognitive Load in the Recognition of Phishing Emails. Front. Big Data 2020, 3, 546860. [Google Scholar] [CrossRef] [Scilit]
- Yang, R.; Zheng, K.; Wu, B.; Li, D.; Wang, Z.; Wang, X. Predicting User Susceptibility to Phishing Based on Multidimensional Features. Comput. Intell. Neurosci. 2022, 2022, 7058972. [Google Scholar] [CrossRef] [Scilit]
- Vytal, K.; Hamann, S. Neuroimaging Support for Discrete Neural Correlates of Basic Emotions: A Voxel-Based Meta-Analysis. J. Cogn. Neurosci. 2010, 22, 2864–2885. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Wicker, B.; Keysers, C.; Plailly, J.; Royet, J.P.; Gallese, V.; Rizzolatti, G. Both of Us Disgusted in My Insula: The Common Neural Basis of Seeing and Feeling Disgust. Neuron 2003, 40, 655–664. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ghous, G.; Najam, S.; Alshehri, M.; Alshahrani, A.; AlQahtani, Y.; Jalal, A.; Liu, H. Attention-Driven Emotion Recognition in EEG: A Transformer-Based Approach With Cross-Dataset Fine-Tuning. IEEE Access 2025, 13, 69369–69394. [Google Scholar] [CrossRef] [Scilit]
- Pan, J.; Bai, C. EEG-Based Emotion Recognition via Convolutional Transformer with Class Confusion-Aware Attention. In Proceedings of the Annual Meeting of the Cognitive Science Society; Curran Associates, Inc.: New York, NY, USA, 2024; Volume 46, Available online: https://escholarship.org/uc/item/21p105jn (accessed on 23 February 2026).
- Shen, X.; Gan, R.; Wang, K.; Yang, S.; Zhang, Q.; Liu, Q.; Zhang, D.; Song, S. Dynamic-Attention-Based EEG State Transition Modeling for Emotion Recognition. arXiv 2024, arXiv:2411.04568. [Google Scholar] [CrossRef] [Scilit]
- Li, G.; Ouyang, D.; Yang, L.; Li, Q.; Tian, K.; Wu, B.; Guo, G. Cross-Subject EEG Linear Domain Adaption Based on Batch Normalization and Depthwise Convolutional Neural Network. Knowl.-Based Syst. 2023, 280, 111011. [Google Scholar] [CrossRef] [Scilit]
- Xu, L.; Ma, Z.; Meng, J.; Xu, M.; Jung, T.-P.; Ming, D. Improving Transfer Performance of Deep Learning with Adaptive Batch Normalization for Brain–Computer Interfaces. In Proceedings of the 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico City, Mexico, 1–5 November 2021; pp. 5800–5803. [Google Scholar] [CrossRef] [Scilit]
- Bajaj, V.; Taran, S.; Sengur, A. Emotion Classification Using Flexible Analytic Wavelet Transform for Electroencephalogram Signals. Health Inf. Sci. Syst. 2018, 6, 12. [Google Scholar] [CrossRef] [Scilit]
- Aliramezani, M.; Farrokhi, A.; Constantinidis, C.; Daliri, M.R. Protocol for Phase-Amplitude Coupling Analysis in Local Field Potentials from Macaque Monkeys to Investigate Neural Oscillation Dynamics. STAR Protoc. 2025, 6, 103877. [Google Scholar] [CrossRef] [Scilit]
- Thant, A.M.; Panitanarak, T. Emotion Recognition Through Advanced Signal Fusion and Kolmogorov-Arnold Networks. IEEE Access 2025, 13, 93259–93270. [Google Scholar] [CrossRef] [Scilit]
- Klimesch, W. An Algorithm for the EEG Frequency Architecture of Consciousness and Brain Body Coupling. Front. Hum. Neurosci. 2013, 7, 766. [Google Scholar] [CrossRef] [Scilit]
- Liu, Z.; Wang, Y.; Vaidya, S.; Ruehle, F.; Halverson, J.; Soljačić, M.; Hou, T.Y.; Tegmark, M. KAN: Kolmogorov–Arnold Networks. arXiv 2025, arXiv:2404.19756. [Google Scholar]
- Bosch, L.T.; Mulder, K.; Boves, L. Phase Synchronization Between EEG Signals as a Function of Differences Between Stimuli Characteristics. In Proceedings of the Interspeech 2019; ISCA: Graz, Austria, 2019; pp. 1213–1217. [Google Scholar] [CrossRef] [Scilit]
- Smith, E.E.; Bel-Bahar, T.S.; Kayser, J. A Systematic Data-Driven Approach to Analyze Sensor-Level EEG Connectivity: Identifying Robust Phase-Synchronized Network Components Using Surface Laplacian with Spectral-Spatial PCA. Psychophysiology 2022, 59, e14080. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Cruz, G.G.; Renczes, B.; Runacres, M.C.; Decuyper, J. State-Space Kolmogorov Arnold Networks for Interpretable Nonlinear System Identification. IEEE Control Syst. Lett. 2025, 9, 847–852. [Google Scholar] [CrossRef] [Scilit]
- Liu, W.; Qiu, J.-L.; Zheng, W.-L.; Lu, B.-L. Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition. IEEE Trans. Cogn. Dev. Syst. 2022, 14, 715–729. [Google Scholar] [CrossRef] [Scilit]
- Mane, M. An EEG Recordings Dataset for Mental Stress Detection. Mendeley Data 2023. [Google Scholar] [CrossRef]
- Singh, K.; Ahirwal, M.K.; Pandey, M. Selected Channel Based Multiclass Emotion Classification from Wearable Human Brain EEG Signal. Meas. Sens. 2025, 39, 101874. [Google Scholar] [CrossRef] [Scilit]
- Apicella, A.; Arpaia, P.; Isgrò, F.; Mastrati, G.; Moccaldi, N. A Survey on EEG-Based Solutions for Emotion Recognition with a Low Number of Channels. IEEE Access 2022, 10, 117411–117428. [Google Scholar] [CrossRef] [Scilit]
- Ahirwal, M.K.; Kose, M.R. Audio-Visual Stimulation Based Emotion Classification by Correlated EEG Channels. Health Technol. 2020, 10, 7–23. [Google Scholar] [CrossRef] [Scilit]
- Wang, Z.M.; Hu, S.Y.; Song, H. Channel Selection Method for EEG Emotion Recognition Using Normalized Mutual Infor-mation. IEEE Access 2019, 7, 143303–143311. [Google Scholar] [CrossRef] [Scilit]
- Xiang, J.Z.; Wang, Q.Y.; Fang, Z.B.; Esquivel, J.A.; Su, Z.X. A Multi-Modal Deep Learning Approach for Stress Detection Using Physiological Signals. Front. Physiol. 2025, 16, 1584299. [Google Scholar] [CrossRef] [Scilit]
- Fernandez, J.; Martínez, R.; Innocenti, B.; López, B. Contribution of EEG Signals for Students’ Stress Detection. IEEE Trans. Affect. Comput. 2024, 16, 1235–1246. [Google Scholar] [CrossRef] [Scilit]
- Roy, B.; Malviya, L.; Kumar, R.; Mal, S.; Kumar, A.; Bhowmik, T.; Hu, J.W. Hybrid Deep Learning Approach for Stress Detection Using Decomposed EEG Signals. Diagnostics 2023, 13, 1936. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Marcolin, F.; Olivetti, E.C.; Castiblanco Jimenez, I.A.; Passavanti, G.; Moos, S.; Vezzetti, E.; Celeghin, A. Stress Assessment with EEG and Machine Learning in Affective VR Environments. Neurocomputing 2025, 638, 130185. [Google Scholar] [CrossRef] [Scilit]
- Sanchez-Vivanco, J.; Hernandez-Alvarez, M. EEG-Based Machine Learning for Emotional Stress Recognition in the Valence–Arousal Space. Ingén. Syst. Inf. 2025, 30, 2739–2746. [Google Scholar] [CrossRef] [Scilit]
- Liu, J.; Wang, C.; Feng, Z.; Zhang, H.; Ma, Y.; Li, H. Electroencephalogram Signals Emotion Recognition Based on Arti-fact-Robust Processing in Speech Tasks. Front. Aging Neurosci. 2022, 14, 945024. [Google Scholar] [CrossRef] [Scilit]
- Immanuel, R.; Skb, S. Advancing Emotion Recognition via EEG Signals Using a Deep Learning Approach with Ensemble Model. J. Intell. Fuzzy Syst. 2024, 47, 143–154. [Google Scholar] [CrossRef] [Scilit]
- Horoi, S.; Orozco Camacho, A.M.; Belilovsky, E.; Wolf, G. Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis. arXiv 2024, arXiv:2407.05385. Available online: https://arxiv.org/abs/2407.05385 (accessed on 1 March 2026). [CrossRef] [Scilit]
- Pabst, S.; Brand, M.; Wolf, O.T. Stress and Decision Making: A Few Minutes Make All the Difference. Behav. Brain Res. 2013, 250, 39–45. [Google Scholar] [CrossRef] [Scilit]
- Giles, G.E.; Mahoney, C.R.; Brunyé, T.T.; Taylor, H.A.; Kanarek, R.B. Stress Effects on Mood, HPA Axis, and Autonomic Response: Comparison of Three Psychosocial Stress Paradigms. PLoS ONE 2014, 9, e113618. [Google Scholar] [CrossRef] [Scilit]
- Bian, W.; Zhang, X.; Dong, Y. Autonomic Nervous System Response Patterns of Test-Anxious Individuals to Evaluative Stress. Front. Psychol. 2022, 13, 824406. [Google Scholar] [CrossRef] [Scilit]














| State | Emotional Manifestations | Characteristic EEG Signs |
|---|---|---|
| Happiness | Joy, satisfaction, a feeling of upliftment | Increased alpha activity in the occipital and parietal regions, frontal asymmetry in favor of the left hemisphere |
| Sadness | Sadness, decreased motivation, introspection | Increased alpha activity in the right frontal lobe, decreased beta, possible increase in theta rhythm |
| Fear | A sense of threat, anxious anticipation | A sharp increase in beta activity (especially high-frequency), a decrease in alpha, increased coherence of frontal–limbic connections |
| Disgust | Rejection, desire to avoid the irritant | Increased theta and beta rhythms in the anterior regions, activity in the insular cortex, moderate frontal asymmetry |
| Neutral | Calmness, absence of strong emotions | Balanced power across ranges, moderate alpha activity in the back of the head, no sharp spikes |
| Approach | Interpretability | Parameters | Feature Type | Cross-Subject Generalizability | Key Limitation |
|---|---|---|---|---|---|
| CNN (shallow) | Low | Medium | Raw/filtered | Moderate | No frequency decomposition |
| CNN-LSTM | Low | High | Temporal | Good | Computationally expensive |
| Transformer | Very low | Very high | Attention-based | Good | Requires large datasets |
| KAN-based (ours) | High | Low–Medium | Wavelet/spline | Good | Novel, limited benchmarks |
| Method | Year | Dataset | Feature Type | Architecture | Evaluation Protocol | Best Accuracy | Key Limitation |
|---|---|---|---|---|---|---|---|
| EEGNet [15] | 2018 | BCI Competition IV-2a | Raw/filtered | Depthwise CNN | Typically subject-dependent/within-subject | 73.55% | No native time–frequency decomposition |
| DeepConvNet [16] | 2017 | BCI Competition IV-2a | Raw temporal | Deep CNN | Typically subject-dependent/within-subject | 60.15% | High parameter count |
| FBCSP [35] | 2012 | BCI Competition IV-2a | Filter bank CSP | CSP + SVM | Subject-dependent | 80% | Handcrafted features |
| STGATE [18] | 2023 | SEED | Graph + TF features | GNN + Transformer | Subject-independent (LOSO on SEED) | 90.37% (SEED, LOSO) | Requires electrode topology |
| CD-FTA [45] | 2025 | SEED-V | Temporal–frequency | CNN + attention | Subject-independent | 90.0% (SEED-V, reported) | No KAN integration |
| CSET-CCA [48] | 2024 | SEED-V | Temporal | CCA-based | Subject-independent | 82.06% (SEED-V) | Limited feature space |
| DAEST [49] | 2025 | SEED-V | DE features | Domain adaptation | Subject-independent | 73.6% (SEED-V) | Poor performance on SEED-V |
| ERTNet [17] | 2024 | SEED-V | Raw temporal | Transformer-based | Subject-independent | 67.17% ± 1.70% (SEED-V) | Requires large data; low SEED-V accuracy |
| WS-KAN-EEGNet (ours) | 2026 | SEED-V | Wavelet/ST + KAN | Ensemble (1D + 2D) | Subject-independent 5-fold CV | 91.3% (SEED-V, this work) | Novel; limited external benchmarks |
| Wavelet Type | Formula | Peculiarities |
|---|---|---|
| Mexican Hat | Good time localization and edge detection | |
| Morlet Wavelet | Balances temporal and frequency localization; effective for rhythmic EEG data. | |
| Derivative of Gaussian | Sensitive to sudden changes; captures signal gradients well. | |
| Meyer Wavelet | Smooth and differentiable; useful for general signal analysis. |
| Scenario | Featured Channels | Citation |
|---|---|---|
| 4 Channels | CP1, Pz, PO4, O1 | [61,62] |
| 5 Channels | AF3, AF4, F3, F4, Pz | [59] |
| 6 Channels | FP1, FP2, F3, F4, P3, P4 | [61,63] |
| 8 Channels | F3, F4, FC5, FC6, C3, C4, P7, P8 | [61,63] |
| Metric | Condition | Happiness | Sadness | Fear | Disgust | Neutral |
|---|---|---|---|---|---|---|
| Mean softmax confidence | Before AdaBN | 0.71 | 0.68 | 0.74 | 0.66 | 0.78 |
| Mean softmax confidence | After AdaBN | 0.83 | 0.79 | 0.86 | 0.77 | 0.88 |
| Metric | Before AdaBN | After AdaBN | ||||
| MMD (SEED-V → TSST feature space) | 0.342 | 0.198 | ||||
| Parameter | SEED-V | Mendeley Stress (TSST) |
|---|---|---|
| Subjects | 16 (10 F, 6 M) | 20+ |
| Age range | 19–24 years | Not specified (adults) |
| EEG system | ESI NeuroScan (62 ch.) | EMOTIV Insight (5 ch.) |
| Channels used | 8 (F3,F4,FC5,FC6,C3,C4,P7,P8) | 5 (AF3,AF4,F3,F4,Pz) |
| Sampling rate | ~200 Hz | ~128 Hz |
| Total trials | 720 (45/subject Г—16 subjects) | Variable (~20 min/session) |
| Trials per class | 144 (balanced) | N/A (continuous) |
| Emotion classes | 5 (Happy, Sad, Fear, Disgust, Neutral) | Stress/Non-stress (mapped to 5 via model) |
| Stimulus type | Emotional video clips (2–4 min) | TSST, Stroop, arithmetic |
| Recording format | .cnt | .csv/.edf |
| Role in study | Training + evaluation (5-fold CV) | Inference only (no fine-tuning) |
| NO | Component | Login → Logout | Description |
|---|---|---|---|
| 1 | Input (Raw EEG) | B × 8 × L → B × 8 × L | Normalized raw EEG signal, 8 channels along the length L. |
| 2 | MultiScaleWaveletBlock (5-path Conv1D Kolmogorov Wavelet) | B × 8 × L → B × 80 × L | Five parallel Conv1DKolWavelets with kernels 3,5,7,9,15 (8 → 16 channels each), channel concatenation (5 × 16), BatchNorm1d and ReLU. |
| 3 | ResidualWaveletBlocks 1–3 + SE | B × 80 × L → B × 128 × L | Cascade of three residual blocks with Conv1DKolWavelet и SE: (1) 80 → 64, k = 5, BN, SE(64), skip Conv1d 80 → 64; (2) 64 → 96, k = 5, BN, SE(96), skip Conv1d 64 → 96; (3) 96 → 128, k = 3, BN, SE(128), skip Conv1d 96 → 128; after each—summation with shortcut and ReLU. |
| 4 | TemporalAttention | B × 128 × L → B × 128 × L | MultiheadAttention with 8 heads by characteristics 128 (through presentation B × L × 128), residual-coeдинeниe и LayerNorm, then back to B × 128 × L. |
| 5 | Global pooling | B × 128 × L → B × 128 | AdaptiveAvgPool1d(1) along the time axis and squeeze dimension L. |
| 6 | DropKAN mask | B × 128 → B × 128 | Stochastic zeroing of features with drop_rate = 0.4 and scaling 1/(1 − 0.4) |
| 7 | Linear + ReLU | B × 128 → B × 64 | Fully connected layer 128 → 64 with activation ReLU. |
| 8 | Linear out (logits_eeg) | B × 64 → B × 5 | Output fully connected layer 64 → 5, generating logits for 5 classes. |
| NO | Component | Login → Logout | Description |
|---|---|---|---|
| 1 | Input (EEG time–frequency maps) | B × 8 × 64 × 64 → B × 8 × 64 × 64 | Normalized 8-channel EEG images (frequency-spatial maps). |
| 2 | STransform2D | B × 8 × 64 × 64 → B × 8 × 64 × 64 | 2D Stockwell-like transform: FFT on two axes, a set of Gaussian windows across scales, and an inverse FFT, averaged across scales, to enhance frequency-spatial features. |
| 3 | Conv2d(8 → 16,3 × 3) + BatchNorm2d(16) + ReLU | B × 8 × 64 × 64 → B × 16 × 64 × 64 | The first convolutional block extracts low-level local patterns from the S transform maps, normalization stabilizes the distribution of activations. |
| 4 | Conv2d(16 → 32,3 × 3) + BatchNorm2d(32) + ReLU | B × 16 × 64 × 64 → B × 32 × 64 × 64 | Deepens the representation by encoding more complex spatial frequency features in the second convolutional block. |
| 5 | Conv2d(32 → 48,3 × 3) + BatchNorm2d (48) + ReLU | B × 32 × 64 × 64 → B × 48 × 64 × 64 | The third convolutional block increases the number of channels to 48, forming a high-level feature map before attention. |
| 6 | spatial_attn1: Conv2d(48 → 1, 3 × 3) + ReLU | (B × 48 × 64 × 64 → B × 1 × 64 × 64 | The first step of spatial attention: convolution of feature maps into a single-channel saliency map. |
| 7 | spatial_attn2: Conv2d(1 → 1, 3 × 3)+ Sigmoid | B × 1 × 64 × 64 → B × 1 × 64 × 64 | Refinement of the attention map and normalization of values to the range [0, 1] for subsequent scaling of features. |
| 8 | Element-wise multiplication x*attn | B × 48 × 64 × 64 → B × 48 × 64 × 64 | Applying spatial attention mask to feature maps, suppressing irrelevant regions. |
| 9 | AdaptiveAvgPool2d(1)+ squeeze | B × 48 × 64 × 64 → B × 48 | Global spatial averaging pooling compresses feature maps into a vector of length 48 for subsequent head KAN. |
| 10 | KANLayer(48 → 32) + ReLU | B × 48→ B × 32 | The first fully connected KAN layer with a base linear part and a spline component on the edges, modeling nonlinear one-dimensional functions. |
| 11 | KANLayer(32 → 16) + ReLU | B × 32→ B × 16 | A second KAN layer that combines and compacts features, allowing for more flexible approximation of complex dependencies than a regular MLP. |
| 12 | Linear(16 → 5) | B × 16→ B × 5 | The final fully connected classifier layer produces 5-class logits for the 2D branch. |
| NO | Component | Login → Logout | Description |
|---|---|---|---|
| 1 | Input logits of branches | logits_img (B × 5), logits_eeg (B × 5) → are passed to the module | Logits of two pre-trained and frozen models 1D-EEG WKAN branch and 2D-ST-KANCNN branch; gradients on their parameters are not calculated. |
| 2 | Trainable scalars αimg, αeeg | no external input → αimg,αeeg | Two nn.Parameters, initialized to 0.5, are learned during ensemble training and specify the relative importance of branches. |
| 3 | Softmax by weight | [αimg,αeeg] → [wimg,weeg] | A vector of two scalars is fed into softmax, resulting in normalized coefficients wimg, weeg with a sum of 1, used as ensemble weights. |
| 4 | Weighted sum of logits | logits_img, logits_eeg, wimg,weeg → logits_ens (B × 5) | The final logits are calculated as logits_ens = wimg⋅logits_img + weeg⋅logits_eeg; it is this tensor that is fed into the loss function and the subsequent softmax by classes. |
| Fusion Strategy | Accuracy | Parameters |
|---|---|---|
| Equal average (0.5/0.5) | 0.901 | 0 |
| Softmax-weighted (ours) | 0.913 | 2 |
| Sigmoid gating | 0.908 | 12 |
| Attention over logits | 0.910 | 30 |
| Category | Parameter | Value |
|---|---|---|
| Training | Optimizer | Adam |
| Learning rate | 1 × 10−3 | |
| Weight decay | 1 × 10−4 | |
| Batch size | 64 | |
| Max epochs | 100 (branches), 30 (ensemble) | |
| Early stopping patience | 15 epochs | |
| Early stopping monitor | Validation loss | |
| Loss function | Weighted cross-entropy | |
| 1D Branch (1D-EEG WKAN) | Kernel sizes | [3, 5, 7, 9, 15] |
| Channel progression | 8 → 80 → 64 → 96 → 128 | |
| Attention heads | 8 | |
| DropKAN rate | 0.4 | |
| SE reduction ratio | 16 | |
| 2D Branch (2D-ST-KANCNN) | Input map size | 64 × 64 |
| Channel progression | 8 → 16 → 32 → 48 | |
| KAN hidden dims | 48 → 32 → 16 → 5 | |
| KAN-specific | B-spline order (k) | 3 |
| Number of grid intervals | 8 | |
| Grid range | [−2, 2] | |
| Wavelet type | Morlet (cmor2.0-1.0) | |
| Wavelet bandwidth (B) | 2.0 | |
| Wavelet center freq (C) | 1.0 | |
| Ensemble | Fusion parameters | 2 (α_img, α_eeg) |
| Initial weights | 0.5, 0.5 | |
| Converged weights | 0.57, 0.43 |
| NO | Matrix | Accuracy | Precision | Recall | F1 |
|---|---|---|---|---|---|
| 1 | 1D-EEG WKAN | 0.874 | 0.876 | 0.873 | 0.876 |
| 2 | 2D-ST-KANCNN | 0.889 | 0.890 | 0.890 | 0.889 |
| 3 | WS-KAN-EEGNet | 0.913 | 0.916 | 0.914 | 0.914 |
| 4 | CNN-1D [58] | 0.823 | 0.821 | 0.823 | 0.822 |
| 5 | CNN-1D-KAN-Wavelet [52] | 0.866 | 0.867 | 0.867 | 0.866 |
| 6 | CD-FTA [45] | 0.900 | 0.906 | 0.906 | 0.906 |
| 7 | CSET-CCA [46] | 0.822 | 0.823 | 0.822 | 0.822 |
| 8 | DAEST [47] | 0.736 | - | - | - |
| 9 | ERTNet [17] | 0.672 | - | - | - |
| NO | Configuration | Accuracy | F1 | Δ vs. Full |
|---|---|---|---|---|
| 1 | WS-KAN-EEGNet (full) | 0.913 | 0.914 | — |
| 2 | 1D-EEG WKAN only | 0.874 | 0.876 | −3.9% |
| 3 | 2D-ST-KANCNN only | 0.889 | 0.889 | −2.4% |
| 4 | 1D branch w/o KAN (standard Conv1D) | 0.841 | 0.839 | −7.2% |
| 5 | 2D branch w/o KAN (standard FC) | 0.862 | 0.860 | −5.1% |
| 6 | Ensemble with equal weights (0.5/0.5) | 0.901 | 0.902 | −1.2% |
| Wavelet | Accuracy | F1 | Key Characteristic |
|---|---|---|---|
| Morlet (cmor2.0-1.0) | 0.874 | 0.876 | Complex modulation captures phase; balanced time–frequency resolution |
| Mexican Hat | 0.858 | 0.860 | Good time localization; misses phase information |
| DOG | 0.851 | 0.853 | Sensitive to transients; limited frequency selectivity |
| Meyer | 0.863 | 0.865 | Smooth, differentiable; good general-purpose performance |
| Phase | TSST Stage | Phishing Analog | Cognitive Process | EEG Signature |
|---|---|---|---|---|
| 1. Threat onset | Anticipation of social evaluation | Receipt of threatening message | Threat perception, amygdala activation | β-rise, α-drop |
| 2. Fear plateau | Public speech under evaluation | Processing urgency/loss cues | Attentional tunneling, working memory overload | Sustained high β, elevated θ |
| 3. Decision under stress | Arithmetic task under pressure | Clicking link/entering credentials | Executive function failure, impulsive action | Peak β/θ, minimal α |
| Dimension | TSST (Social-Evaluative Stress) | Phishing (Decision Stress) |
|---|---|---|
| Primary pathway | HPA axis: cortisol release via social-evaluative threat | Sympathetic–adrenal–medullary (SAM): catecholamine surge via urgency/threat framing |
| Cognitive bias | Self-focused attention, social comparison | Loss aversion, authority compliance, scarcity framing |
| EEG signature | Sustained β-increase, α-suppression, frontal θ-increase | Expected: similar β/α pattern + enhanced P300 suppression under time pressure |
| Temporal profile | Slow onset (minutes), sustained plateau, slow recovery | Rapid onset (seconds), spike during message processing, quick resolution or escalation |
| Common mechanism | Prefrontal executive function suppression → weakened cognitive control | Same downstream effect via different upstream triggers |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Pleshakova, E.; Osipov, A.; Yudin, A.; Gataullin, S. EEG-Based Emotion Dynamics Recognition Using Hybrid AI Models for Cybersecurity. Technologies 2026, 14, 209. https://doi.org/10.3390/technologies14040209
Pleshakova E, Osipov A, Yudin A, Gataullin S. EEG-Based Emotion Dynamics Recognition Using Hybrid AI Models for Cybersecurity. Technologies. 2026; 14(4):209. https://doi.org/10.3390/technologies14040209
Chicago/Turabian StylePleshakova, Ekaterina, Aleksey Osipov, Alexander Yudin, and Sergey Gataullin. 2026. "EEG-Based Emotion Dynamics Recognition Using Hybrid AI Models for Cybersecurity" Technologies 14, no. 4: 209. https://doi.org/10.3390/technologies14040209
APA StylePleshakova, E., Osipov, A., Yudin, A., & Gataullin, S. (2026). EEG-Based Emotion Dynamics Recognition Using Hybrid AI Models for Cybersecurity. Technologies, 14(4), 209. https://doi.org/10.3390/technologies14040209

