Search Results (55)

Remote photoplethysmography (rPPG) enables non-contact physiological measurement for emotion recognition, yet the temporally sparse nature of emotional cardiovascular responses, intrinsic measurement noise, weak session-level labels, and subtle correlates of valence pose critical challenges. To address these issues, we propose a physiologically inspired deep learning framework comprising a Multi-scale Temporal Dynamics Encoder (MTDE) to capture autonomic nervous system dynamics across multiple timescales, an adaptive sparse α-Entmax attention mechanism to identify salient emotional segments amidst noisy signals, Gated Temporal Pooling for the robust aggregation of emotional features, and a structured three-phase curriculum learning strategy to systematically handle temporal sparsity, weak labels, and noise. Evaluated on the MAHNOB-HCI dataset (27 subjects and 527 sessions with a subject-mixed split), our temporal-only model achieved competitive performance in arousal recognition (66.04% accuracy; 61.97% weighted F1-score), surpassing prior CNN-LSTM baselines. However, lower performance in valence (62.26% accuracy) revealed inherent physiological limitations regarding a unimodal temporal cardiovascular analysis. These findings establish clear benchmarks for temporal-only rPPG emotion recognition and underscore the necessity of incorporating spatial or multimodal information to effectively capture nuanced emotional dimensions such as valence, guiding future research directions in affective computing. Full article

Remote vital sensing in veterinary medicine is a relatively new area of practice, which involves the acquisition of data without invasion of the body cavities of live animals. This paper aims to review several technologies in remote vital sensing: infrared thermography, remote photoplethysmography (rPPG), radar, wearable sensors, and computer vision and machine learning. In each of these technologies, we outline its concepts, uses, strengths, and limitations in multiple animal species, and its potential to reshape health surveillance, welfare evaluation, and clinical medicine in animals. The review also provides information about the problems associated with applying these technologies, including species differences, external conditions, and the question of the reliability and classification of these technologies. Additional topics discussed in this review include future developments such as the use of artificial intelligence, combining different sensing methods, and creating monitoring solutions tailored to specific animal species. This contribution gives a clear understanding of the status and future possibilities of remote vital sensing in veterinary applications and stresses the importance of that technology for the development of the veterinary field in terms of animal health and science. Full article

Advancements in camera technology over the past two decades have made image-based monitoring increasingly accessible for healthcare applications. Imaging photoplethysmography (iPPG) and remote photoplethysmography (rPPG) are non-invasive methods for measuring vital signs, such as heart rate, respiratory rate, oxygen saturation, and blood pressure, without physical contact. rPPG utilizes basic cameras to detect physiological changes, while rPPG enables remote monitoring by capturing subtle skin colour variations linked to blood flow. Various rPPG techniques, including colour-based, motion-based, multispectral, and depth-based approaches, enhance accuracy and resilience. These technologies are beneficial not only for healthcare but also for fitness tracking, stress management, and security systems, offering a promising future for contactless physiological monitoring. In this article, there is an overview of these methods and their uniqueness for use in remote photoplethysmography. Full article

This study explores the extraction of remote Photoplethysmography (rPPG) signals from images using various neural network architectures, addressing the challenge of accurate signal estimation in biomedical contexts. The objective is to evaluate the effectiveness of different models in capturing rPPG signals from dataset snapshots. Two training strategies were investigated: pre-training models with only the fully connected layer being fine-tuned and training the entire network from scratch. The analysis reveals that models trained from scratch consistently outperform their pre-trained counterparts in extracting rPPG signals. Among the architectures assessed, DenseNet121 demonstrated superior performance, offering the most reliable results in this context. These findings underscore the potential of neural networks in advancing rPPG signal extraction, which has promising applications in fields such as clinical monitoring and personalized medical care. This study contributes to the integration of advanced imaging techniques and neural network-based analysis in biomedical engineering, paving the way for more robust and efficient methodologies. Full article

With the rise of modern healthcare monitoring, heart rate (HR) estimation using remote photoplethysmography (rPPG) has gained attention for its non-contact, continuous tracking capabilities. However, most HR estimation methods rely on stable, fixed sampling intervals, while practical image capture often involves irregular frame rates and missing data, leading to inaccuracies in HR measurements. This study addresses these issues by introducing low-complexity timing correction methods, including linear, cubic, and filter interpolation, to improve HR estimation from rPPG signals under conditions of irregular sampling and data loss. Through a comparative analysis, this study offers insights into efficient timing correction techniques for enhancing HR estimation from rPPG, particularly suitable for edge-computing applications where low computational complexity is essential. Cubic interpolation can provide robust performance in reconstructing signals but requires higher computational resources, while linear and filter interpolation offer more efficient solutions. The proposed low-complexity timing correction methods improve the reliability of rPPG-based HR estimation, making it a more robust solution for real-world healthcare applications. Full article

A solution to address the issues of environmental light interference in Remote Photoplethysmography (rPPG) methods is proposed in this paper. First, signals from the face’s region of interest (ROI) and background noise signals are simultaneously collected, and the two signals are processed by a differential to obtain a more accurate rPPG signal. This method effectively suppresses background noise and enhances signal quality. Secondly, the singular spectrum analysis algorithm (SSA) is enhanced to further improve the accuracy of heart rate detection. The algorithm’s parameters are adaptively optimized by integrating the spectral and periodic characteristics of the heart rate signal. Experimental results demonstrate that the method proposed in this paper effectively mitigates the effects of lighting changes on heart rate detection, thereby enhancing detection accuracy. Overall, the experiments indicate that the proposed method significantly improves the effectiveness and accuracy of heart rate detection, achieving a high level of consistency with existing contact-based detection methods. Full article

Remote photoplethysmography (rPPG) has attracted growing attention due to its non-contact nature. However, existing non-contact heart rate detection methods are often affected by noise from motion artifacts and changes in lighting, which can lead to a decrease in detection accuracy. To solve this problem, this paper initially employs manual extraction to precisely define the facial Region of Interest (ROI), expanding the facial area while avoiding rigid regions such as the eyes and mouth to minimize the impact of motion artifacts. Additionally, during the training phase, illumination normalization is employed on video frames with uneven lighting to mitigate noise caused by lighting fluctuations. Finally, this paper introduces a 3D convolutional neural network (CNN) method incorporating an attention mechanism for heart rate detection from facial videos. We optimize the traditional 3D-CNN to capture global features in spatiotemporal data more effectively. The SimAM attention mechanism is introduced to enable the model to precisely focus on and enhance facial ROI feature representations. Following the extraction of rPPG signals, a heart rate estimation network using a bidirectional long short-term memory (BiLSTM) model is employed to derive the heart rate from the signals. The method introduced here is experimentally validated on two publicly available datasets, UBFC-rPPG and PURE. The mean absolute errors were 0.24 bpm and 0.65 bpm, the root mean square errors were 0.63 bpm and 1.30 bpm, and the Pearson correlation coefficients reached 0.99, confirming the method’s reliability. Comparisons of predicted signals with ground truth signals further validated its accuracy. Full article

Due to its non-contact characteristics, remote photoplethysmography (rPPG) has attracted widespread attention in recent years, and has been widely applied for remote physiological measurements. However, most of the existing rPPG models are unable to estimate multiple physiological signals simultaneously, and the performance of the limited available multi-task models is also restricted due to their single-model architectures. To address the above problems, this study proposes MultiPhys, adopting a heterogeneous network fusion approach for its development. Specifically, a Convolutional Neural Network (CNN) is used to quickly extract local features in the early stage, a transformer captures global context and long-distance dependencies, and Mamba is used to compensate for the transformer’s deficiencies, reducing the computational complexity and improving the accuracy of the model. Additionally, a gate is utilized for feature selection, which classifies the features of different physiological indicators. Finally, physiological indicators are estimated after passing features to each task-related head. Experiments on three datasets show that MultiPhys has superior performance in handling multiple tasks. The results of cross-dataset and hyper-parameter sensitivity tests also verify its generalization ability and robustness, respectively. MultiPhys can be considered as an effective solution for remote physiological estimation, thus promoting the development of this field. Full article

Remote photo-plethysmography (rPPG) is a useful camera-based health motioning method that can measure the heart rhythm from facial videos. Many well-established deep learning models can provide highly accurate and robust results in measuring heart rate (HR) and heart rate variability (HRV). However, these methods are unable to effectively eliminate illumination variation and motion artifact disturbances, and their substantial computational resource requirements significantly limit their applicability in real-world scenarios. Hence, we propose a lightweight multi-frequency network named MFF-Net to measure heart rhythm via facial videos in a short time. Firstly, we propose a multi-frequency mode signal fusion (MFF) mechanism, which can separate the characteristics of different modes of the original rPPG signals and send them to a processor with independent parameters, helping the network recover blood volume pulse (BVP) signals accurately under a complex noise environment. In addition, in order to help the network extract the characteristics of different modal signals effectively, we designed a temporal multiscale convolution module (TMSC-module) and spectrum self-attention module (SSA-module). The TMSC-module can expand the receptive field of the signal-refining network, obtain more abundant multiscale information, and transmit it to the signal reconstruction network. The SSA-module can help a signal reconstruction network locate the obvious inferior parts in the reconstruction process so as to make better decisions when merging multi-dimensional signals. Finally, in order to solve the over-fitting phenomenon that easily occurs in the network, we propose an over-fitting sampling training scheme to further improve the fitting ability of the network. Comprehensive experiments were conducted on three benchmark datasets, and we estimated HR and HRV based on the BVP signals derived by MFF-Net. Compared with state-of-the-art methods, our approach achieves better performance both on HR and HRV estimation with lower computational burden. We can conclude that the proposed MFF-Net has the opportunity to be applied in many real-world scenarios. Full article

Remote photoplethysmography (rPPG) is a non-contact technology that monitors heart activity by detecting subtle color changes within the facial blood vessels. It provides an unconstrained and unconscious approach that can be widely applied to health monitoring systems. In recent years, research has been actively conducted to improve rPPG signals and to extract significant information from facial videos. However, rPPG can be vulnerable to degradation due to changes in the illumination and motion of a subject, and overcoming these challenges remains difficult. In this study, we propose a machine learning-based filter bank (MLFB) noise reduction algorithm to improve the quality of rPPG signals. The MLFB algorithm determines the optimal spectral band for extracting information on cardiovascular activity and reconstructing an rPPG signal using a support vector machine. The proposed approach was validated with an open dataset, achieving a 35.5% (i.e., resulting in a mean absolute error of 2.5 beats per minute) higher accuracy than those of conventional methods. The proposed algorithm can be integrated into various rPPG algorithms for the pre-processing of RGB signals. Moreover, its computational efficiency is expected to enable straightforward implementation in system development, making it broadly applicable across the healthcare field. Full article

Non-contact heart monitoring is crucial in advancing telemedicine, fitness tracking, and mass screening. Remote photoplethysmography (rPPG) is a non-contact technique to obtain information about heart pulse by analyzing the changes in the light intensity reflected or absorbed by the skin during the blood circulation cycle. However, this technique is sensitive to environmental lightning and different skin pigmentation, resulting in unreliable results. This research presents a multimodal approach to non-contact heart rate estimation by combining facial video and physical attributes, including age, gender, weight, height, and body mass index (BMI). For this purpose, we collected local datasets from 60 individuals containing a 1 min facial video and physical attributes such as age, gender, weight, and height, and we derived the BMI variable from the weight and height. We compare the performance of two machine learning models, support vector regression (SVR) and random forest regression on the multimodal dataset. The experimental results demonstrate that incorporating a multimodal approach enhances model performance, with the random forest model achieving superior results, yielding a mean absolute error (MAE) of 3.057 bpm, a root mean squared error (RMSE) of 10.532 bpm, and a mean absolute percentage error (MAPE) of 4.2% that outperforms the state-of-the-art rPPG methods. These findings highlight the potential for interpretable, non-contact, real-time heart rate measurement systems to contribute effectively to applications in telemedicine and mass screening. Full article

Non-contact heart rate monitoring from facial videos utilizing remote photoplethysmography (rPPG) has gained significant traction in remote health monitoring. Given that rPPG captures the dynamic blood flow within the human body and constitutes a time-series signal characterized by periodic properties, this study introduced a three-dimensional convolutional neural network (3D CNN) designed to simultaneously address long-term periodic and short-term temporal characteristics for effective rPPG signal extraction. Firstly, differential operations are employed to preprocess video data, enhancing the face’s dynamic features. Secondly, building upon the 3D CNN framework, multi-scale dilated convolutions and self-attention mechanisms were integrated to enhance the model’s temporal modeling capabilities further. Finally, interpolation techniques are applied to refine the heart rate calculation methodology. The experiments conducted on the UBFC-rPPG dataset indicate that, compared with the existing optimal algorithm, the average absolute error (MAE) and the root mean square error (RMSE) realized significant enhancements of approximately 28% and 35%. Additionally, through comprehensive analyses such as cross-dataset experiments and complexity analyses, the validity and stability of the proposed algorithm in the task of heart rate estimation were manifested. Full article

Remote photoplethysmography (rPPG) refers to a non-contact technique that measures heart rate through analyzing the subtle signal changes of facial blood flow captured by video sensors. It is widely used in contactless medical monitoring, remote health management, and activity monitoring, providing a more convenient and non-invasive way to monitor heart health. However, factors such as ambient light variations, facial movements, and differences in light absorption and reflection pose challenges to deep learning-based methods. To solve these difficulties, we put forward a measurement network of heart rate based on multiscale features. In this study, we designed and implemented a dual-branch signal processing framework that combines static and dynamic features, proposing a novel and efficient method for feature fusion, enhancing the robustness and reliability of the signal. Furthermore, we proposed an alternate time-shift module to enhance the model’s temporal depth. To integrate the features extracted at different scales, we utilized a multiscale feature fusion method, enabling the model to accurately capture subtle changes in blood flow. We conducted cross-validation on three public datasets: UBFC-rPPG, PURE, and MMPD. The results demonstrate that MDAR not only ensures fast inference speed but also significantly improves performance. The two main indicators, MAE and MAPE, achieved improvements of at least 30.6% and 30.2%, respectively, surpassing state-of-the-art methods. These conclusions highlight the potential advantages of MDAR for practical applications. Full article

Heart rate (HR) is an important indicator reflecting the overall physical and mental health of the human body, playing a crucial role in diagnosing cardiovascular and neurological diseases. Recent research has revealed that variations in the light absorption of human skin captured through facial video over the cardiac cycle, due to changes in blood volume, can be utilized for non-contact HR estimation. However, most existing methods rely on single-modal video sources (such as RGB or NIR), which often yield suboptimal results due to noise and the limitations of a single information source. To overcome these challenges, this paper proposes a multimodal information fusion architecture named the spatiotemporal sensitive network (SS-Net) for non-contact heart rate estimation. Firstly, spatiotemporal feature maps are utilized to extract physiological signals from RGB and NIR videos effectively. Next, a spatiotemporal sensitive (SS) module is introduced to extract useful physiological signal information from both RGB and NIR spatiotemporal maps. Finally, a multi-level spatiotemporal context fusion (MLSC) module is designed to fuse and complement information between the visible light and infrared modalities. Then, different levels of fused features are refined in task-specific branches to predict both remote photoplethysmography (rPPG) signals and heart rate (HR) signals. Experiments conducted on three datasets demonstrate that the proposed SS-Net achieves superior performance compared to existing methods. Full article

Background and Objectives: Sepsis, a leading global health challenge, accounts for around 20% of deaths worldwide. The complexity of sepsis, especially the difference between bacterial and viral etiologies, requires an effective assessment of microcirculation during resuscitation. This study aimed to evaluate the impact of infusion therapy on microcirculation in patients with sepsis, focusing on bacterial- and COVID-19-associated sepsis using remote photoplethysmography (rPPG) and the automated capillary refill time (aCRT). Materials and Methods: This single-center prospective study was conducted in the ICU of Pauls Stradins Clinical University Hospital, including 20 patients with sepsis/septic shock. The patients were selected based on hemodynamic instability and divided into COVID-19 and Bacterial Septic Shock groups. Fluid responsiveness was assessed using the Passive Leg Raising Test (PLRT). Systemic hemodynamics and microcirculation were monitored through MAP CRT, rPPG, and serum lactate levels. Statistical analyses compared responses within and between the groups across different stages of the protocol. Results: The Bacterial group exhibited higher initial serum lactate levels and more pronounced microcirculatory dysfunction than the COVID-19 group. rPPG was more sensitive in detecting perfusion changes, showing significant differences between the groups. The automated CRT demonstrated greater sensitivity compared to the manual CRT, revealing significant differences during PLRT stages between bacterial- and COVID-19-associated sepsis. Both groups had a transient hemodynamic response to PLRT, with subsequent stabilization upon fluid infusion. Conclusions: When managing patients with sepsis in intensive care, monitoring microcirculation is of paramount importance in infusion therapy. Our study highlights the potential of rPPG and aCRT as tools for this purpose. These techniques can be used in conjunction with routine parameters, such as lactate levels and systemic hemodynamic parameters, to provide a comprehensive assessment of a patient’s condition. Full article