Search Results (253)

Background: Emotional prosody refers to the variations in pitch, pause, melody, rhythm, and stress of pronunciation conveying emotional meaning during speech. Although several studies demonstrated that the cerebellum is involved in the network subserving recognition of emotional facial expressions, there is only preliminary evidence suggesting its possible contribution to recognising emotional prosody by modulating the activity of cerebello-prefrontal circuits. The present study aims to further explore the role of the left and right cerebellum in the recognition of emotional prosody in a sample of healthy individuals who were required to identify emotions (happiness, anger, sadness, surprise, disgust, and neutral) from vocal stimuli selected from a validated database (EMOVO corpus). Methods: Anodal transcranial Direct Current Stimulation (tDCS) was used in offline mode to modulate cerebellar activity before the emotional prosody recognition task, and functional near-infrared spectroscopy (fNIRS) was used to monitor stimulation-related changes in oxy- and deoxy- haemoglobin (O2HB and HHB) in prefrontal areas (PFC). Results: Right cerebellar stimulation reduced reaction times in the recognition of all emotions (except neutral and disgust) as compared to both the sham and left cerebellar stimulation, while accuracy was not affected by the stimulation. Haemodynamic data revealed that right cerebellar stimulation reduced O2HB and increased HHB in the PFC bilaterally relative to the other stimulation conditions. Conclusions: These findings are consistent with the involvement of the right cerebellum in modulating emotional processing and in regulating cerebello-prefrontal circuits. Full article

This paper presents a novel deep neural network for real-time emotion recognition based on facial expression measurement, optimized for low computational complexity, called Lightweight Expression Recognition Network (LiExNet). The LiExNet architecture comprises only 42,000 parameters and integrates convolutional layers, depthwise convolutional layers, an efficient channel attention mechanism, and fully connected layers. The network was trained and evaluated on three widely used datasets (CK+, KDEF, and FER2013) and a custom dataset, EMOTION-ITCH. This dataset comprises facial expressions from both industrial workers and non-workers, enabling the study of emotional responses to occupational stress. Experimental results demonstrate that LiExNet achieves high recognition performance with minimal computational resources, reaching 99.5% accuracy on CK+, 88.2% on KDEF, 79.2% on FER2013, and 96% on EMOTION-ITCH. In addition, LiExNet supports real-time inference on embedded systems, requiring only 0.03 MB of memory and 1.38 GFLOPs of computational power. Comparative evaluations show that among real-time methods, LiExNet achieves the best results, ranking first on the CK+ and KDEF datasets, and second on FER2013, demonstrating consistent performance across these datasets. These results position LiExNet as a practical and robust alternative for real-time emotion monitoring and emotional dissonance assessment in occupational settings, including hardware-constrained and embedded environments. Full article

To ensure the effectiveness of online teaching, educators must understand students’ learning progress. This study proposes LWKD-ViT, a framework designed to accurately capture students’ emotions during online courses. The framework is built on a lightweight facial expression recognition (FER) model with modifications to the fusion block. In addition, knowledge distillation (KD) is integrated into the online course platform to enhance performance. The framework follows a defined process involving face detection, tracking, and clustering to extract facial sequences for each student. An improved model, MobileViT-Local, developed by the authors, extracts emotion features from individual frames of students’ facial video streams for classification and prediction. Students’ facial images are captured through their device cameras and analyzed in real time on their devices, eliminating the need to transmit videos to the teacher’s computer or a remote server. To evaluate the performance of MobileViT-Local, comprehensive tests were conducted on benchmark datasets, including RAFD, RAF-DB, and FER2013, as well as a self-built dataset, SCAUOL. Experimental results demonstrate the model’s competitive performance and superior efficiency. Due to the use of knowledge distillation, the proposed model achieves a prediction accuracy of 94.96%, surpassing other mainstream models. It also exhibits excellent performance, with optimal FLOPs of 0.265 G and a compact size of 4.96 M, while maintaining acceptable accuracy. Full article

A core challenge in multimodal emotion recognition lies in the precise capture of the inherent multimodal interactive nature of human emotions. Addressing the limitation of existing methods, which often process visual signals (facial expressions) and physiological signals (EEG, ECG, EOG, and GSR) in isolation and thus fail to exploit their complementary strengths effectively, this paper presents a new multimodal emotion recognition framework called the Gated Biological Visual Network (GBV-Net). This framework enhances emotion recognition accuracy through deep synergistic fusion of facial expressions and physiological signals. GBV-Net integrates three core modules: (1) a facial feature extractor based on a modified ConvNeXt V2 architecture incorporating lightweight Transformers, specifically designed to capture subtle spatio-temporal dynamics in facial expressions; (2) a hybrid physiological feature extractor combining 1D convolutions, Temporal Convolutional Networks (TCNs), and convolutional self-attention mechanisms, adept at modeling local patterns and long-range temporal dependencies in physiological signals; and (3) an enhanced gated attention fusion module capable of adaptively learning inter-modal weights to achieve dynamic, synergistic integration at the feature level. A thorough investigation of the publicly accessible DEAP and MAHNOB-HCI datasets reveals that GBV-Net surpasses contemporary methods. Specifically, on the DEAP dataset, the model attained classification accuracies of 95.10% for Valence and 95.65% for Arousal, with F1-scores of 95.52% and 96.35%, respectively. On MAHNOB-HCI, the accuracies achieved were 97.28% for Valence and 97.73% for Arousal, with F1-scores of 97.50% and 97.74%, respectively. These experimental findings substantiate that GBV-Net effectively captures deep-level interactive information between multimodal signals, thereby improving emotion recognition accuracy. Full article

Learning and understanding of art are increasingly understood as dynamic processes in which emotion and cognition unfold over time. However, classroom-based evidence on how structured temporal intervals and guided prompts reshape students’ emotional experience remains limited. This study addresses these gaps by quantitatively examining changes in emotion over time in a higher education institution. Employing a comparative experimental design, third-year undergraduate art students participated in two structured courses, where emotional responses were captured using an emotion recognition approach (facial expression and self-reported text) during two sessions: initial impression and delayed impression (three days later). The findings reveal a high consistency in dominant facial expressions and substantial agreement in self-reported emotions across both settings. However, the delayed impression elicited greater emotional diversity and intensity, reflecting deeper cognitive engagement and emotional processing over time. These results reveal a longitudinal trajectory of emotion influenced by guided reflective re-view over time. Emotional dynamics extend medium theory by embedding temporal and affective dimensions into TBMA course settings. This study proposes an ethically grounded and technically feasible framework for emotion recognition that supports reflective learning rather than mere measurement. Together, these contributions redefine TBMA education as a temporal and emotional ecosystem and provide an empirical foundation for future research on how emotion fosters understanding, interest, and appreciation in higher media art education. Full article

Micro-expression recognition (MER) aims to detect brief and subtle facial movements that reveal suppressed emotions, discerning authentic emotional responses in scenarios such as visitor experience analysis in museum settings. However, it remains a highly challenging task due to the fleeting duration, low intensity, and limited availability of annotated data. Most existing approaches rely solely on either appearance or motion cues, thereby restricting their ability to capture expressive information fully. To overcome these limitations, we propose a lightweight multi-modal fusion network, termed M³ENet, which integrates both motion and appearance cues through early-stage feature fusion. Specifically, our model extracts horizontal, vertical, and strain-based optical flow between the onset and apex frames, alongside RGB images from the onset, apex, and offset frames. These inputs are processed by two modality-specific subnetworks, whose features are fused to exploit complementary information for robust classification. To improve generalization in low data regimes, we employ targeted data augmentation and adopt focal loss to mitigate class imbalance. Extensive experiments on five benchmark datasets, including CASME I, CASME II, CAS(ME)², SAMM, and MMEW, demonstrate that M³ENet achieves state-of-the-art performance with high efficiency. Ablation studies and Grad-CAM visualizations further confirm the effectiveness and interpretability of the proposed architecture. Full article

Background/Objectives: Infant emotional responses during complementary feeding offer key insights into early developmental processes and feeding behaviors. AI-driven facial emotion analysis presents a novel, objective method to quantify these subtle expressions, potentially informing interventions in early childhood nutrition. We aimed to investigate how maternal and infant traits influence infants’ emotional responses during complementary feeding using an automated facial analysis tool. Methods: This multi-center study involved 117 typically developing infants (6–11 months) and their mothers. Standardized feeding sessions were recorded, and OpenFace software quantified six emotions (surprise, sadness, fear, happiness, anger, disgust). Data were normalized and analyzed via Generalized Estimating Equations to identify associations with maternal BMI, education, work status, and infant age, sex, and complementary feeding initiation. Results: Emotional responses did not differ significantly across five food groups. Infants of mothers with BMI > 30 kg/m² showed greater surprise, while those whose mothers were well-educated and not working displayed more happiness. Older infants and those introduced to complementary feeding before six months exhibited higher levels of anger. Parental or infant food selectivity did not significantly affect responses. Conclusions: The findings indicate that maternal and infant demographic factors exert a more pronounced influence on infant emotional responses during complementary feeding than the type of food provided. These results highlight the importance of integrating broader psychosocial variables into early feeding practices and underscore the potential utility of AI-driven facial emotion analysis in advancing research on infant development. Full article

The influence of lighting color on food is a multidimensional process, linking visual interventions with people’s perception of food appearance, physiological responses, and psychological associations. This study, as a preliminary exploratory research, aims to initially investigate the effects of different lighting colors on food-induced consumer appetite and emotional perception. By measuring consumers’ physiological facial expression data, we verify whether the results are consistent with self-reported subjective evaluations. Questionnaires, Shapiro–Wilk tests, and one-sample t-tests were employed for data mining and cross-validation and combined with generalized facial expression recognition (GFER) technology to analyze participants’ emotional perceptions under various lighting colors. The results show that consumers displayed the most positive emotions and the highest appetite under 2700 K warm white light. Under this condition, the average intensity of participants’ “happy” emotion was 0.25 (SD = 0.12), indicating a clear positive emotional state. Eating willingness also reached its peak at 2700 K. In contrast, blue light-induced negative emotions and lower appetite. Among all lighting types, blue light evoked the strongest “sad” emotion (M = 0.39). This study provides a preliminary exploration of the theoretical framework regarding the relationship between food and consumer behavior, offering new perspectives for product marketing in the food industry and consumer food preference cognition. However, the generalizability of its conclusions still requires further verification in subsequent studies. Full article

Traditional police interrogation processes remain largely time-consuming and reliant on substantial human effort for both analysis and documentation. Intuition Artificial Intelligence (INTU-AI) is a Windows application designed to digitalize the administrative workflow associated with police interrogations, while enhancing procedural efficiency through the integration of AI-driven emotion recognition models. The system employs a multimodal approach that captures and analyzes emotional states using three primary vectors: Facial Expression Recognition (FER), Speech Emotion Recognition (SER), and Text-based Emotion Analysis (TEA). This triangulated methodology aims to identify emotional inconsistencies and detect potential suppression or concealment of affective responses by interviewees. INTU-AI serves as a decision-support tool rather than a replacement for human judgment. By automating bureaucratic tasks, it allows investigators to focus on critical aspects of the interrogation process. The system was validated in practical training sessions with inspectors and with a 12-question questionnaire. The results indicate a strong acceptance of the system in terms of its usability, existing functionalities, practical utility of the program, user experience, and open-ended qualitative responses. Full article

Advances in multimodal artificial intelligence enable new sensor-inspired approaches to lie detection by combining behavioral perception with generative reasoning. This study presents a deception detection framework that integrates deep video and audio processing with large language models guided by chain-of-thought (CoT) prompting. We interpret neural architectures such as ViViT (for video) and HuBERT (for speech) as digital behavioral sensors that extract implicit emotional and cognitive cues, including micro-expressions, vocal stress, and timing irregularities. We further incorporate a GPT-5-based prompt-level fusion approach for video–language–emotion alignment and zero-shot inference. This method jointly processes visual frames, textual transcripts, and emotion recognition outputs, enabling the system to generate interpretable deception hypotheses without any task-specific fine-tuning. Facial expressions are treated as high-resolution affective signals captured via visual sensors, while audio encodes prosodic markers of stress. Our experimental setup is based on the DOLOS dataset, which provides high-quality multimodal recordings of deceptive and truthful behavior. We also evaluate a continual learning setup that transfers emotional understanding to deception classification. Results indicate that multimodal fusion and CoT-based reasoning increase classification accuracy and interpretability. The proposed system bridges the gap between raw behavioral data and semantic inference, laying a foundation for AI-driven lie detection with interpretable sensor analogues. Full article

In recent years, most conventional emotion recognition approaches have concentrated primarily on facial cues, often overlooking complementary sources of information such as body posture and contextual background. This limitation reduces their effectiveness in complex, real-world environments. In this work, we present a multi-branch emotion recognition framework that separately processes facial, bodily, and contextual information using three dedicated neural networks. To better capture contextual cues, we intentionally mask the face and body of the main subject within the scene, prompting the model to explore alternative visual elements that may convey emotional states. To further enhance the quality of the extracted features, we integrate both channel and spatial attention mechanisms into the network architecture. Evaluated on the challenging NCAER-S dataset, our model achieves an accuracy of 56.42%, surpassing the state-of-the-art GLAMOUR-Net. These results highlight the effectiveness of combining multi-cue representation and attention-guided feature extraction for robust emotion recognition in unconstrained settings. The findings also highlight the importance of accurate emotion recognition for human–computer interaction, where affect detection enables systems to adapt to users and deliver more effective experiences. Full article

In a world where artificial intelligence is constantly changing education, taking students’ feelings into account is a crucial framework for enhancing their engagement and academic performance. This article presents LearnerEmotions, an online application that employs machine vision technology to determine how learners are feeling in real time through their facial expressions. Teachers and institutions can access analytical dashboards and monitor students’ emotions with this tool, which is designed for use in both in-person and remote classes. The facial expression recognition model used in this application achieved an average accuracy of 0.91 and a loss of 0.3 in the real environment. More than 9 million emotional data points were gathered from an experiment involving 65 computer engineering students, and these insights were correlated with attendance and academic performance. While negative emotions like anger, sadness, and fear are associated with decreased performance and lower attendance, the statistical study shows a strong correlation between positive feelings like surprise and joy and successful academic performance. These results underline the necessity of technological tools that offer immediate pedagogical regulation and support the notion that emotions play an important role in the learning process. Thus, LearnerEmotions, which considers students’ emotional states, is a potential first step toward more adaptive learning. Full article

This paper presents a real-time smart mirror system combining multiple AI modules for multimodal health monitoring. The proposed platform integrates three core components: facial expression analysis, remote photoplethysmography (rPPG), and conversational AI. A key innovation lies in transforming the Moving Average rPPG (MA-rPPG) model—originally developed for offline batch processing—into a real-time, continuously streaming setup, enabling seamless heart rate and peripheral oxygen saturation (SpO₂) monitoring using standard webcams. The system also incorporates the DeepFace facial analysis library for live emotion, age detection, and a Generative Pre-trained Transformer 4o (GPT-4o)-based mental health chatbot with bilingual (English/Korean) support and voice synthesis. Embedded into a touchscreen mirror with Graphical User Interface (GUI), this solution delivers ambient, low-interruption interaction and real-time user feedback. By unifying these AI modules within an interactive smart mirror, our findings demonstrate the feasibility of integrating multimodal sensing (rPPG, affect detection) and conversational AI into a real-time smart mirror platform. This system is presented as a feasibility-stage prototype to promote real-time health awareness and empathetic feedback. The physiological validation was limited to a single subject, and the user evaluation constituted only a small formative assessment; therefore, results should be interpreted strictly as preliminary feasibility evidence. The system is not intended to provide clinical diagnosis or generalizable accuracy at this stage. Full article

The rise of virtual models in the digital age presents a new frontier for cosmetic advertising. Nevertheless, the comparative effectiveness of “idealized” versus “naturalistic” facial features in these models remains a topic of debate and an area of development. This study examines the impact of “idealized” and “naturalistic” facial features in virtual models on consumers’ cognitive and affective responses. Using eye-tracking and a structural equation model, we analyzed visual attention patterns and the roles of affective resonance, trustworthiness, likability, and expertise perception. The results indicate that non-homogeneous or defective naturalistic features increase visual attention and purchase intention, with consumers focusing on imperfections such as freckles. In contrast, idealized facial features mainly draw attention to areas such as the eyes and nose. Mediation analysis reveals that likability and affective resonance are primary influences on purchase intention, while expertise perception and trustworthiness are secondary. This experiment suggests that consumers prioritize socio-emotional connections over professional authority when evaluating naturalistic designs. Our findings provide a framework for virtual model design, helping brands balance aesthetics with psychological optimization, and offer insights into the interplay between visual stimuli and human cognitive and emotional processes in decision-making. Full article

Borderline personality disorder (BPD) is associated with alterations in emotion processing. To date, no study has tested automatic emotion perception under conditions of unawareness of emotion stimuli. We administered a priming paradigm based on facial expressions and measured judgmental and gaze behavior during an evaluation task. A total of 31 patients with BPD and 31 non-patients (NPs) viewed a briefly shown emotional (angry, fearful, sad, or happy) or neutral face followed by a neutral facial expression (target). Areas of interest (AOI) were the eyes and the mouth. All participants included in our analysis were subjectively unaware of the emotional primes. Concerning evaluative ratings, no prime effects were observed. For early dwell time, a significant interaction between prime category and AOI was found. Both BPD patients and NPs dwelled longer on the eyes after the presentation of angry and fearful primes than of happy primes and dwelled longer on the mouth after the presentation of happy primes than of sad and neutral primes. Patients rated target faces more negatively. BPD patients, when compared to NPs, seem not to show alterations in automatic attention orienting due to covert facial emotions. Regardless of primes, individuals with BPD seem to be characterized by an increased negative interpretation of neutral facial expressions. Full article