Search Results (348)

Emotion recognition (ER) in human–computer interaction (HCI) holds immense potential for real-world applications, but traditional approaches based on electroencephalography (EEG) face challenges due to the complexity and impracticality of collecting and analyzing EEG data in ambulatory settings. This study explores electrodermal activity (EDA), a simpler measure of the sympathetic nervous system response that can be collected at multiple peripheral body sites, as a potential alternative for ER. We investigated the variable frequency complex demodulation (VFCDM) technique to analyze EDA and EEG signals and used deep learning models (ResNet50 and MobileNetV2) to classify arousal states (high arousal, HA vs. low arousal, LA). Our results show that EDA signals analyzed by VFCDM and classified by MobileNetV2 achieve promising performance, with an accuracy of 91.45%, comparable to the best EEG-based model (91.98%), in arousal classification. This suggests that EDA offers a viable and more practically accessible approach to ER in HCI compared to traditional EEG-based methods. Future work should explore larger and more diverse datasets, incorporate valence classification through multimodal fusion, and investigate the neural mechanisms underlying EDA-EEG interactions during emotional processing to further advance robust ER for HCI applications. Full article

Consumer-grade wearable sensors may enable continuous monitoring of pilot workload and stress during flight training, yet most prior studies rely on simulators, raw-score labelling, and within-subject validation, limiting generalisability. This study evaluates whether electrodermal activity (EDA), electrocardiogram (ECG)-derived features, and wrist skin temperature, recorded from an Empatica Embrace Plus and a Polar H10 during real Cessna 172 flight training, can classify pilots’ task-relative workload and stress deviations. Thirty-five pilots completed four flight segments and rated workload and stress after each. Fold-safe two-way residual binary labels removed inter-pilot scale-use differences and task-level effects, and five classifiers were evaluated under leave-one-subject-out (LOSO) cross-validation with Benjamini–Hochberg FDR correction. Under LOSO, a Linear SVC on combined features classified stress (macro F1 = 0.607) and XGBoost on EDA classified workload (macro F1 = 0.598) significantly above chance ($p_{\mathrm{adj}}=0.033$); both remained stable under nested cross-validation with an inner hyperparameter search (nested 0.606 and 0.561). A LightGBM model on EDA gave a numerically higher stress score (0.611) that did not survive nested validation. Subject-dependent within-subject validation produced higher apparent performance (macro F1 = 0.853 for stress and 0.791 for workload), but a stricter within-pilot analysis was unstable. These contrasts indicate that personalised classification may be feasible after calibration, whereas uncalibrated cross-pilot prediction in real flight remains modest, with post-flight debriefing the most plausible near-term application. Full article

Accurate stress monitoring is critical for high-risk professions like firefighting, yet existing wearable solutions face challenges balancing accuracy with practical usability. While electrodermal activity (EDA) offers a non-invasive, single-sensor approach, current automated feature extraction methods fail to capture stress-discriminative patterns effectively. We developed a hybrid stress detection pipeline combining 20 hand-crafted physiological features with 32 deep-learned features from a supervised convolutional autoencoder. Unlike traditional unsupervised approaches optimized solely for signal reconstruction, our architecture employs a dual-head design with weighted classification loss to guide feature learning toward stress discrimination. The system was validated on the WESAD dataset (15 subjects) using rigorous leave-one-subject-out (LOSO) cross-validation, along with comprehensive preprocessing, including cvxEDA decomposition, adaptive artifact detection, and physiological peak validation. Our optimized K-Nearest Neighbors classifier achieved 98.62% accuracy, surpassing the industry-standard PyEDA benchmark (97.0%) by 1.62 percentage points. The model demonstrated 97.58% sensitivity (true positive rate) and 98.92% specificity (true negative rate), with only 2.42% false negatives—critical for safety-critical applications. Ablation studies revealed that unsupervised autoencoder features alone achieved only 55% accuracy, increasing to 89% with supervised learning and 98.62% with the hybrid approach, representing a 43.62-percentage-point improvement. This work demonstrates that combining domain-specific physiological knowledge with label-aware deep learning produces more discriminative features than either approach alone. The resulting system successfully translates complex probabilistic outputs into an interpretable 1–10 stress score, providing a practical foundation for real-time stress monitoring in wearable devices for first responders. Full article

This study investigates how chromatic manipulation of cinematic content modulates emotional engagement, with specific attention to sex-differentiated responses. We used a mixed factorial design with chromatic condition as a within-subject factor and biological sex as a between-subject factor, counterbalanced across scenes through a 3 × 3 Latin square that renders scene identity orthogonal to chromatic condition by construction. Thirty adult viewers were recorded with synchronised facial-expression analysis (AFFDEX 5.1), blink detection, and galvanic skin response (Shimmer GSR). The primary inferential target was the Condition × Sex interaction on automated positive facial valence. This interaction was statistically reliable under three converging tests: a mixed-effects model (${\beta }_{\mathrm{Mod}\times \mathrm{F}}=-4.48$, $SE=2.16$, 95% CI $[-8.81,-0.14]$, $p=0.043$), a participant-level cluster bootstrap (2000 resamples; 95% percentile CI $[-9.78,-0.63]$; $p_{\mathrm{boot}}=0.011$), and a label-permutation test. The effect was stable under leave-one-subject-out resampling (100% sign-stability) and persisted after introducing scene as a fixed factor. Blink rate and electrodermal activation showed directionally consistent but weaker interaction patterns. A multidimensional engagement framework that separates attentional-autonomic intensity from expressive valence supports interpretation of the finding as specific to expressive affective behavior rather than to overall activation. The results provide empirical evidence that chromatic manipulation in realistic cinematic stimuli modulates expressive affective responses in a sex-dependent manner, and they establish a reproducible multimodal biometric framework for chromatic impact assessment. Full article

Multimodal emotion recognition is challenging because behavioral signals and physiological responses evolve at different temporal rates. Facial expressions and speech often change rapidly after an emotional event, whereas peripheral biosignals such as electrodermal activity, blood volume pulse, and skin temperature exhibit delayed and smoother dynamics. This temporal inconsistency can degrade fusion performance, particularly in real-world recordings with noisy or missing modalities. To address this issue, this study proposes SGMT, an S-PACE Gated Multimodal Transformer for emotion recognition using speech, facial video, and physiological signals. The proposed SGMT introduces S-PACE, a physiology-guided cross-attention mechanism that aligns fast behavioral cues with slower biosignal representations without assuming a fixed temporal delay. A Quality-Aware Gate further improves robustness by adaptively weighting modalities according to signal reliability. The fused representations are processed using a Temporal Swin Transformer and a Perceiver Fusion module for arousal–valence prediction and emotion quadrant classification. Experiments are conducted on the Korean multimodal emotion datasets KEMDy20 and K-EmoCon under different modality settings. SGMT achieves arousal UARs of 68.4% on KEMDy20 and 62.9% on K-EmoCon, with quadrant accuracies of 44.7% and 62.5%, respectively. Ablation studies demonstrate that the proposed alignment and gating strategies provide more stable multimodal fusion than conventional feature concatenation. The results indicate that SGMT effectively adapts to varying modality availability and improves multimodal emotion recognition in naturalistic environments. Full article

With the continued rise in outpatient surgical procedures, modern medicine requires more advanced tools for pain and anxiety monitoring and management. The current standard of care requires patient responses on visual analog scales, which may be subjective and are difficult to assess when a subject is unresponsive. Electrodermal activity (EDA) and pulse rate variability (PRV), two non-invasive, wearable, and objective measurements of sympathetic nervous system activity, can help provide insight into a patient’s psychological or emotional state without user input, allowing for continued monitoring even when a patient is unable to respond. However, methods based on these measurements have largely been relegated to longer duration (>60 s) or post hoc analysis, which does not suit the needs of medical care environments. Here we propose new methods for handling ultra-short (<10 s) signals to allow rapid evaluation of pain and anxiety state. We show how machine learning models trained on these signals can obtain high degrees of classification performance (AUC > 0.88) between no pain or anxiety and medium or higher pain and anxiety on signals obtained during two different forms of painful stimulation. We also show how these signals can measure the degree of stimulation irrespective of perceived pain from the patient. Further development of these algorithms will allow for greater monitoring and control of patient comfort in a clinical setting. Full article

Virtual reality (VR) is increasingly used in public speaking training, yet the distinct roles of environmental context and virtual audience design remain unclear. This study examines how avatar visual style (realistic vs. stylized) and scene type (natural vs. indoor) influence subjective experience and physiological stress. A total of 132 participants were assigned to a 2 × 2 between-subjects experiment. Subjective experience was assessed using standardized questionnaires, while physiological responses were measured via electrodermal activity and heart rate variability, complemented by post-experiment interviews. Results revealed a dissociation between subjective and physiological responses. Natural environments significantly enhanced user satisfaction and overall experience, whereas avatar style primarily influenced physiological stress. Specifically, stylized avatars elicited lower electrodermal activity than realistic avatars, indicating reduced sympathetic arousal. No significant interaction effects were observed. Mediation analyses showed no significant roles of perceived support or threat, suggesting that physiological responses may not rely on explicit cognitive appraisal. Qualitative findings further indicated that ambiguous audience feedback limited evaluative interpretation. These findings support a dual-pathway framework in which environmental context shapes cognitive–affective experience, whereas avatar realism modulates implicit physiological stress. This study provides theoretical insights and practical implications for designing VR systems that enhance user comfort and reduce stress. Full article

Driving with assistive devices creates complex cognitive and emotional demands that require systematic investigation. This study uses a multivariate approach based on subjective and objective measures to evaluate mental workload (MWL), stress and emotional state during simulated driving with an assistive device. Thirty healthy adults ($42\pm 13$ years of age, 7 females) completed four driving tasks combining two levels of difficulty (Easy vs. Hard) and two steering tools (wheel vs. single-pin aid). Subjective measures from NASA Task Load Index and Self-Assessment Manikin were collected, as well as physiological parameters from electroencephalographic, electrocardiographic, and electrodermal activity signals. The results revealed that the assistive device significantly induced increases in perceived physical demand, frustration, loss of emotional control and stress, yet reducing intrinsic sympathetic response represented by electrodermal activity parameters. Multivariate analyses highlighted that combining different physiological predictors improved MWL estimation. This study marks an initial step towards understanding the impact of assistive devices on MWL and stress in post-acute individuals returning to driving. Full article

Wearable devices increasingly capture biosignals such as electrocardiograms, photoplethysmograms, inertial signals, and electrodermal activity during daily life, enabling earlier detection and continuous monitoring outside the clinic. Real-time edge machine learning can convert these streams into timely, privacy-preserving inference by placing computation on a wearable (device-only) or a paired phone, with intermittent cloud assist used selectively for dashboards, summarisation, and lifecycle management. Clinical adoption remains uneven because free-living data are noisy, labels are often delayed, and device ecosystems evolve over time. This narrative review organises the literature as an end-to-end deployment pathway: sensing and artefact management, streaming windowing and multimodal alignment, and model families suited to on-device inference. We compare classical feature-based pipelines with learned representations, including compact CNN/TCN and recurrent and efficient attention-based models, and discuss when self-supervised pretraining and distillation are most useful in low-label settings. We then synthesise deployment engineering levers (quantisation, pruning, and distillation) and benchmarking requirements, emphasising runtime constraints that determine feasibility: latency per update, peak RAM, energy per inference, duty cycle, and thermal behaviour. Applications are grouped across cardiovascular monitoring, blood pressure and haemodynamics, sleep and respiration, and movement and stress, with explicit attention to false-alert burden, adherence, and workflow integration. To support translation, we provide a validation ladder and a reliability toolkit covering calibration, uncertainty-aware thresholds and deferral, drift monitoring triggers, and safe update governance. The novelty of this review is a deployment-oriented synthesis that ties modelling choices to edge tiers and resource budgets and provides reusable reporting templates, including an edge-cost card and comparative tables spanning modalities, models, deployment levers, applications, and reliability requirements. Full article

Background/Objectives: The aim of this pilot study was to evaluate the feasibility of developing individualized machine learning models using nocturnal wearable-derived autonomic nervous system (ANS) and sleep metrics to predict next-day headache risk in patients with migraine. We also examined the associations between nocturnal ANS and sleep measures and patient-reported outcome measures (PROMs) related to nociplastic pain, migraine burden, and non-restorative sleep (NRS). Methods: Adults with migraine wore the wrist-worn Empatica EmbracePlus^® wearable during sleep and completed daily headache diaries for approximately 4 weeks (N = 10). Participants also completed daily headache diaries and PROMs assessing nociplastic pain, migraine burden, and non-restorative sleep. Personalized machine learning (ML) models were developed to predict next-day headache using nocturnal ANS activity (e.g., pulse rate variability (PRV), electrodermal activity (EDA), respiratory rate (RR)) and sleep metrics (e.g., interruptions, duration, awakenings). Model performance was evaluated using area under the receiver operating characteristic and precision–recall curves (AUROC, AUPRC), sensitivity, specificity, accuracy, and precision. Spearman correlations assessed the relationship between wearable-derived metrics and patient-reported outcome measurements of sleep quality (PROMIS-Fatigue, PROMIS-Sleep Disturbance) and a surrogate marker of nociplastic pain (Fibromyalgia (FM) Score). Results: 9 out of 10 participants wore the EmbracePlus device for at least the target duration of four weeks. For the next-day headache prediction, model performance varied between individuals; area under the ROC curve (AUROC) ranged from 28.2% to 81.2%. Nocturnal measures of EDA were strongly correlated with the FM score (Spearman’s rho = 0.72–0.75, p < 0.05). Conclusions: Phasic EDA may warrant further investigation as a potential physiological indicator related to nociplastic pain mechanisms and next-day headache. However, these findings are preliminary, and larger multicenter trials are needed to confirm results of this pilot study. Full article

Real-time psychological stress detection on wearable and edge devices requires models that are accurate, computationally efficient, and small enough for on-device deployment. This paper proposes a Micro-Attention CNN Hybrid Architecture for stress recognition using wearable bio-signals. The model uses six sensor channels, namely tri-axial acceleration, electrodermal activity, heart rate, and skin temperature, and classifies three stress levels: no stress, low stress, and high stress. This study is conducted on a public wearable sensor dataset collected from 15 nurses during hospital work, providing a realistic benchmark for continuous stress monitoring under practical conditions. The proposed architecture combines one-dimensional and depthwise separable convolutions with a lightweight attention module to emphasize the most informative temporal patterns in short multivariate signal segments. To support deployment on resource-constrained devices, we further apply structured pruning, selective quantization-aware training, and post-training quantization. The full-precision model achieves a Macro-F1 score of 99.63%, while the final compressed model retains 98.03% Macro-F1 with a model size of 1.76 kilobytes and a CPU inference latency of 0.40 ms. Additional analyses show that most residual errors occur near the boundary between low stress and neighboring classes, while simple post-compression calibration improves reliability. These results demonstrate that accurate and low-latency stress detection using wearable bio-signals is feasible on compact edge hardware without transmitting raw sensor streams off-device. Full article

Background: Digital learning platforms increasingly leverage semantic web technologies to support interoperable and adaptive e-learning. However, the usability and cognitive impact of web-based authoring tools are still mainly assessed through subjective questionnaires and interaction logs, which provide limited time resolution and weak diagnostic power for identifying specific interface bottlenecks. Methods: We propose a multimodal evaluation of SOULSS, a semantic web-oriented platform for creating and optimizing digital learning contents. Eighteen participants completed an authoring workflow organized into three macro-segments (tutorial, initialization, module creation) while wearable electroencephalography, electrodermal activity, photoplethysmography, and eye tracking were recorded; objective metrics were analyzed both across macro-segments and within predefined micro-activities, whereas subjective engagement was collected after each macro-segment using the UES-SF. Results: Objective measures indicated increased EEG-derived mental workload and stress, higher tonic sympathetic arousal, and greater visual search and interaction effort during initialization and module creation, while UES-SF scores were lower during initialization. Fine-grained analyses localized critical elements to tutorial navigation options, the new course entry point, and spoiler-related controls. Repeated-measures correlations linked subjective scores with objective markers and supported an association between stress-related activation and delayed visual discovery. Conclusions: Integrating neurophysiological and eye tracking measures enables a more diagnostic assessment of semantic web-based authoring platforms than questionnaires alone, providing actionable evidence for iterative UX optimization and supporting a more user-centred design of digital educational tools. Full article

Objective pain assessment remains a significant challenge in clinical and research settings due to the subjective nature of self-reported measures. Physiological signals, particularly electrodermal activity (EDA), have emerged as promising indicators of autonomic responses associated with pain. Although recent advances in deep learning have improved the modelling of complex biosignals, many existing approaches remain computationally demanding, limiting their applicability for real-time monitoring in wearable and embedded systems. This paper proposes a fully convolutional network (FCN) for automated pain recognition using EDA signals. The proposed model is designed to efficiently capture temporal patterns in physiological data while maintaining low computational complexity. The approach is evaluated on the AI4Pain dataset for three-class pain classification (No Pain, Low Pain, High Pain). Experimental results show that the proposed FCN achieves an accuracy of 79.23% in offline evaluation. Furthermore, the model enables real-time inference with a latency of 0.47 ms, achieving 73.14% accuracy during real-time operation. These results demonstrate that convolutional architectures can provide an effective balance between predictive performance and computational efficiency, supporting the development of real-time physiological pain monitoring systems using wearable sensing technologies. Full article

The remote tower operation mode has changed how controllers acquire, integrate, and interpret operational information, making Situation Awareness (SA) prediction more challenging because of the coupling of multiple heterogeneous information sources. To address the limitations of existing physiological-data-based studies in modeling cross-modal relationships and deep multimodal interactions, this study proposes MGRF-Net, a multimodal physiological data-driven model for predicting remote tower controllers’ SA. The model first encodes eye-tracking, electroencephalography, electrocardiography, and electrodermal activity signals independently to obtain high-level temporal representations. A graph attention-enhanced relational learning module is then introduced to capture interactive dependencies among modalities, followed by a dual-branch gated fusion mechanism to adaptively integrate multimodal information and improve prediction stability. Using multimodal physiological data collected from a remote tower simulation experiment and evaluated with 12-fold cross-validation, MGRF-Net achieved 0.0658 RMSE, 0.0461 MAE, 0.8579 R², and 0.9308 PCC, outperforming LightGBM, MLP, PatchTST, iTransformer, and TimeMixer. Ablation experiments and SHAP analysis further confirmed the effectiveness and interpretability of the proposed model. The results indicate that MGRF-Net can effectively capture cross-modal coupling patterns in the formation of controllers’ SA and provides a promising approach for complex cognitive state monitoring and intelligent assistance in remote tower operations. Full article

Precise stress identification in virtual reality (VR) settings continues to be difficult because of thermoregulatory mechanisms that modify electrodermal activity (EDA) independently of emotional responses. This research presents a temperature-corrected framework that distinguishes authentic stress-induced EDA from heat-associated physiological reactions by combining two complementary thermal modeling techniques: a proportionality model and a data-driven adaptive scaling approach. Utilizing the Wearable Emotion Sensing and Detection (WESAD) dataset, temperature variations were synchronized with observed conductance patterns to adjust for thermal distortions that mask stress-specific indicators. The temperature-corrected features enhanced differentiation between stress-related and thermally influenced EDA activities, improving physiological precision and ecological authenticity. Statistical examination revealed strong distinction between affective states in both raw conductance and peripheral temperature measurements. Additionally, the adaptive scaling model produced more distinct condition-specific patterns than the proportionality method. Feature importance findings showed temperature-derived parameters as reliable contributors to classification consistency. These results emphasize temperature compensation as an essential preprocessing procedure for dependable stress identification in VR settings, allowing more accurate interpretation of EDA across different thermal circumstances. Full article