# Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Dataset Description

#### 2.2. iPPG Signal Processing

#### 2.2.1. Region of Interest (ROI) Selection and RGB Extraction

- Face detection: There are three main popular approaches for the ROI choice, namely taking a rectangular ROI encompassing the whole face [11,23,28,29], considering the forehead and cheekbone regions [30,31], or considering only the forehead [32]. Since, in the UBFC-RPPG dataset, hair covers the forehead region for several subjects, our approach considers the whole face region as ROI. To detect the face and select the ROI, a facial rectangle is located for the first frame, employing a cascade classifier constructed using the Viola–Jones algorithm [33].
- Face tracking: To eliminate the problem of rigid head motions where the subject moves their head outside the defined ROI bounding box, a face tracking system is desirable. Given the high computational cost of the Viola–Jones algorithm, we use the Lucas–Kanase approach (KLT tracker) [34], tracking specific features of the face over time.
- Skin masking: Only skin pixels contribute to the PR-related information. Therefore, skin masking is performed on every frame to filter out the nonskin pixels. The RGBH-H-CbCr skin color model [35] is applied: a skin color map is determined based on the skin color distribution and utilized on the chrominance segments of the input frames to distinguish pixels that seem to be skin.
- Raw RGB signal extraction: The time-variant raw RGB signals ${c}^{0}\left(t\right)=({r}^{0}\left(t\right),{g}^{0}\left(t\right),{b}^{0}\left(t\right))$ are produced by calculating the average pixel value of the skin pixels within the ROI for all frames over time. The average color intensities over the ROI frames in time are calculated as follows:$${c}^{0}\left(t\right)={({r}^{0}\left(t\right),{g}^{0}\left(t\right),{b}^{0}\left(t\right))}^{T}=\frac{1}{\left|ROI\right(t\left)\right|}\sum _{(i,j)\u03f5ROI\left(t\right)}{c}_{i,j}\left(t\right)$$

#### 2.2.2. Preprocessing of RGB Signals

#### 2.2.3. iPPG Signal Extraction

- ⯀
- GRD: In the GRD method [17], the green channel is used for the extraction of iPPG, since it generally has the highest signal amplitude [38], whereas the red channel is considered to be an artifact. Thus, the iPPG signal can be computed as follows:$$iPP{G}^{0}\left(t\right)=g\left(t\right)-r\left(t\right)$$
- ⯀
- AGRD: The AGRD method [18] includes an adaptive bandpass filter with the aim to remove residual motion artifacts within the iPPG signal. The approach can be described by the following equation:$$iPP{G}^{0}\left(t\right)=\left|\right|{c}^{0}\left(t\right)\left|\right|(\frac{g\left(t\right)}{{g}^{0}\left(t\right)}-\frac{r\left(t\right)}{{r}^{0}\left(t\right)})$$$$\left|\right|{c}^{0}\left(t\right)\left|\right|=\sqrt{{({\left({r}^{0}\left(t\right)\right)}^{2}+{\left({g}^{0}\left(t\right)\right)}^{2}+{\left({b}^{0}\left(t\right)\right)}^{2})}^{2}}$$It should be noted that preprocessing is an essential step for the AGRD method because, otherwise, $g\left(t\right)={g}^{0}\left(t\right)$ and $r\left(t\right)={r}^{0}\left(t\right)$ will result in a zero iPPG signal.
- ⯀
- PCA: The PCA procedure [19,28,39] is a linear dimensionality reduction technique that identifies patterns in the RGB signals in order to capture intensity variations due to blood pulses. In the PCA, a set of observed signals from correlated variables is projected into a linearly uncorrelated orthogonal basis, called principal components. The principal components are defined by ${s}_{i}={f}_{i}^{T}\xb7X$, where X is the set of observed signals and ${f}_{i}$ are the corresponding eigenvectors of the covariance matrix $C=E\left(X{X}^{T}\right)$. The number of principal components is usually lower than the number of observed multivariate signals.
- ⯀
- ICA: The ICA technique [11,22,40] is the most popular blind source separation technique for iPPG computation. It is used to separate unknown source signals $S\left(t\right)$ from a set of observed mixed signals given by $X\left(t\right)=AS\left(t\right)$, where A is the mixing matrix [14]. The approximated source signals can be found as $\widehat{S}\left(t\right)=Wt$, where W is the separation matrix that approximates the inverse of A. ICA assumes that the components are statistically independent and non-Gaussian and will then choose the component with the most prominent peak in the PR bandwidth.
- ⯀
- CHROM: The CHROM method, as proposed by [23], aims for robustness to subject motion by employing a model of PPG-induced variations in color intensity. For this technique, the iPPG signal is defined as:$$iPP{G}^{0}\left(t\right)={x}_{1}\left(t\right)-\frac{{\sigma}_{1}(t,L)}{{\sigma}_{2}(t,L)}{x}_{2}\left(t\right)$$$${\sigma}_{i}(t,L)=\sqrt{\frac{1}{L-1}\sum _{k=0}^{L-1}{\left({x}_{i}(t-k)\right)}^{2}-\frac{1}{L(L-1)}{\left(\sum _{k=0}^{L-1}{x}_{i}(t-k)\right)}^{2}}$$
- ⯀
- POS: The POS method [13] is quite similar to the CHROM method and can be considered as its simplified and improved version. The iPPG signal is calculated as follows:$$iPP{G}^{0}\left(t\right)={x}_{1}\left(t\right)+\frac{{\sigma}_{1}(t,L)}{{\sigma}_{2}(t,L)}{x}_{2}\left(t\right)$$Here, ${\sigma}_{1}(t,L)$ and ${\sigma}_{2}(t,L)$ are again the L-point running standard deviations of ${x}_{1}\left(t\right)$ and ${x}_{2}\left(t\right)$, respectively. However, now, they are defined as ${x}_{1}\left(t\right)=g\left(t\right)-b\left(t\right)$ and ${x}_{2}\left(t\right)=g\left(t\right)+b\left(t\right)-2r\left(t\right)$. Like in the CHROM method, L corresponds to 1.6 s, as suggested in [13].
- ⯀
- LE: The LE method [20] is a technique aimed at unfolding a nonlinear data distribution in a hyperdimensional space, in order to reduce its dimensionality. When the approach is applied as the iPPG extraction method, it should increase the accuracy in the separation of the iPPG signal from residual sources of fluctuations in light. The LE algorithm maps the averaged RGB signals for the ith frame into a three-dimensional (R-G-B) space, and the final goal is to map their distribution onto a one-dimensional space, preserving the local relationship between data points. Firstly, the adjacent graph G is constructed, computing the Euclidean distance between the data points. Nodes i and j of the graph G are considered adjacent if i is among the k-nearest neighbors of j and vice versa. Then, manifold learning is used to solve the following optimization problem:$$min\sum _{i,j=1}^{n}\left|\right|{y}_{i}-{y}_{j}{\left|\right|}^{2}{W}_{ij}$$
- ⯀
- SPE: SPE [21] is a self-organizing algorithm used to produce low-dimensional embeddings that preserve similarities between a set of related observations. In this study, the SPE approach is applied for the first time to RGB data. In our case, the similarities are the fluctuations in the RGB signal intensities due to blood pulsation, from which the iPPG signal can be estimated. The method starts with an initial configuration, and iteratively refines itself by randomly selecting points ${x}_{a},{x}_{b}$ and adjusting their coordinates to match the Euclidean distances ${d}_{a,b}$ on the map more closely to their respective proximities ${r}_{a,b}$. To avoid oscillatory behavior, the magnitude of the adjustments was controlled by a learning rate parameter, $\lambda $, which decreases during the data point refinement. The refined coordinates are updated by:$${x}_{a}\leftarrow {x}_{a}+\lambda \frac{1}{2}\frac{{r}_{ab}-{d}_{ab}}{{d}_{ab}+\u03f5}({x}_{a}-{x}_{b})$$$${x}_{b}\leftarrow {x}_{b}+\lambda \frac{1}{2}\frac{{r}_{ab}-{d}_{ab}}{{d}_{ab}+\u03f5}({x}_{b}-{x}_{a})$$

#### 2.2.4. Postprocessing of iPPG Signals

**Wavelet filtering:**An adaptive two-step wavelet filtering [13,17,18,41] is applied, assuming that frequency components of the $iPP{G}^{0}$ signal related to noise have weaker power with respect to the components related to the PR. The first step of the method was to perform a continuous wavelet of the $iPP{G}^{0}$ signal. Here, the wavelet coefficients with a wide Gaussian window centered at a scale corresponding to the maximum of squared wavelet coefficients are averaged over a 15 s temporal running window. Secondly, a general Gaussian filter is applied. To reconstruct the iPPG signal, the inverse continuous wavelet is performed [41].**Empirical mode decomposition:**The purpose of empirical mode decomposition (EMD) [42] is to split the $iPP{G}^{0}$ signal into a noise component and PR-related component. The EMD technique decomposes the signal into several unique intrinsic mode functions (IMFs) and one residue function (R) according to:$$g\left(t\right)=\sum _{i=1}^{n}IM{F}_{i}\left(t\right)+{R}_{n}\left(t\right)$$**Outlier suppression:**Am MA filter was again applied to smooth out the signal and suppress the high-frequency peaks that correspond to noise.

#### 2.3. Pulse Rate and Pulse Rate Variability Analysis

#### 2.4. Quality Metrics

## 3. Results

#### 3.1. Spearman Correlation

#### 3.2. Normalized Root Mean Square Error

#### 3.3. Bland–Altman Analysis

## 4. Discussion and Conclusions

## 5. Limitations and Future Work

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

AGRD | Adaptive green–red difference |

ANS | Autonomic nervous system |

CHROME | Chrominace-based |

GRD | Green–red difference |

HR | Heart rate |

HRV | Heart rate variability |

IBI | Interbeat intervals |

ICA | Independent component analysis |

iPPG | Image-based photoplethysmography |

LE | Laplacian eigenmap |

PCA | Principal component analysis |

POS | Plane-orthogonal-to-skin |

PPG | Photoplethysmography |

PR | Pulse rate |

PRV | Pulse rate variability |

RGB | Red-green-blue |

ROI | Region of interest |

SPE | Stochastic proximity embedding |

## References

- Favilla, R.; Zuccala, V.C.; Coppini, G. Heart Rate and Heart Rate Variability from Single-Channel Video and ICA Integration of Multiple Signals. IEEE J. Biomed. Health Inform.
**2019**, 23, 2398–2408. [Google Scholar] [CrossRef] [PubMed] - Acharya, U.R.; Joseph, K.P.; Kannathal, N.; Lim, C.M.; Suri, J.S. Heart rate variability: A review. Med. Biol. Eng. Comput.
**2006**, 44, 1031–1051. [Google Scholar] [CrossRef] [PubMed] - Gil, E.; Orini, M.; Bailón, R.; Vergara, J.M.; Mainardi, L.; Laguna, P. Photoplethysmography pulse rate variability as a surrogate measurement of heart rate variability during non-stationary conditions. Physiol. Meas.
**2010**, 31, 1271–1290. [Google Scholar] [CrossRef] [PubMed] - Yu, S.G.; Kim, S.E.; Kim, N.H.; Suh, K.H.; Lee, E.C. Pulse rate variability analysis using remote photoplethysmography signals. Sensors
**2021**, 21, 6241. [Google Scholar] [CrossRef] - Monkaresi, H.; Bosch, N.; Calvo, R.A.; D’Mello, S.K. Automated Detection of Engagement Using Video-Based Estimation of Facial Expressions and Heart Rate. IEEE Trans. Affect. Comput.
**2017**, 8, 15–28. [Google Scholar] [CrossRef] - Sigari, M.H.; Fathy, M.; Soryani, M. A driver face monitoring system for fatigue and distraction detection. Int. J. Veh. Technol.
**2013**, 2013, 263983. [Google Scholar] [CrossRef] [Green Version] - Sasangohar, F.; Davis, E.; Kash, B.A.; Shah, S.R. Remote Patient Monitoring and Telemedicine in Neonatal and Pediatric Settings: Scoping Literature Review. J. Med. Internet Res.
**2018**, 20, e9403. [Google Scholar] [CrossRef] - Garbey, M.; Sun, N.; Merla, A.; Pavlidis, I. Contact-free measurement of cardiac pulse based on the analysis of thermal imagery. IEEE Trans. Biomed. Eng.
**2007**, 54, 1418–1426. [Google Scholar] [CrossRef] - Boric-Lubecke, O.; Lubecke, V.; Mostafanezhad, I. Amplitude modulation issues in Doppler radar heart signal extraction. In Proceedings of the 2011 IEEE Radio and Wireless Week, RWW 2011—2011 IEEE Topical Conference on Biomedical Wireless Technologies, Networks, and Sensing Systems, BioWireleSS 2011, Phoenix, AZ, USA, 16–20 January 2011; pp. 103–106. [Google Scholar] [CrossRef]
- Mesleh, A.; Skopin, D.; Baglikov, S.; Quteishat, A. Heart Rate Extraction from Vowel Speech Signals. J. Comput. Sci. Technol.
**2012**, 27, 1243–1251. [Google Scholar] [CrossRef] - Poh, M.Z.; McDuff, D.J.; Picard, R.W. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans. Bio-Med. Eng.
**2011**, 58, 7–11. [Google Scholar] [CrossRef] [Green Version] - Pursche, T.; Krajewski, J.; Moeller, R. Video-based heart rate measurement from human faces. In Proceedings of the 2012 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 13–16 January 2012; pp. 544–545. [Google Scholar] [CrossRef]
- Wang, W.; Den Brinker, A.C.; Stuijk, S.; De Haan, G. Algorithmic Principles of Remote PPG. IEEE Trans. Biomed. Eng.
**2017**, 64, 1479–1491. [Google Scholar] [CrossRef] [Green Version] - Sikdar, A.; Behera, S.K.; Dogra, D.P. Computer-vision-guided human pulse rate estimation: A review. IEEE Rev. Biomed. Eng.
**2016**, 9, 91–105. [Google Scholar] [CrossRef] [PubMed] - Unakafov, A.M. Pulse rate estimation using imaging photoplethysmography: Generic framework and comparison of methods on a publicly available dataset. Biomed. Phys. Eng. Express
**2018**, 4, 045001. [Google Scholar] [CrossRef] - Sun, Y.; Thakor, N. Photoplethysmography Revisited: From Contact to Noncontact, from Point to Imaging. IEEE Trans. Biomed. Eng.
**2016**, 63, 463–477. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Hülsbusch, M. An Image-Based Functional Method for Opto-Electronic Detection of Skin-Perfusion; RWTH Aachen: Aachen, Germany, 2008. [Google Scholar]
- Feng, L.; Po, L.M.; Xu, X.; Li, Y.; Ma, R. Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Trans. Circuits Syst. Video Technol.
**2015**, 25, 879–891. [Google Scholar] [CrossRef] - Abdi, H.; Williams, L.J. Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat.
**2010**, 2, 433–459. [Google Scholar] [CrossRef] - Wei, L.; Tian, Y.; Wang, Y.; Ebrahimi, T.; Huang, T. Automatic webcam-based human heart rate measurements using laplacian eigenmap. In Proceedings of the Computer Vision–ACCV 2012: 11th Asian Conference on Computer Vision, Daejeon, Repulic of Korea, 5–9 November 2012; Revised Selected Papers, Part II 11. pp. 281–292. [Google Scholar]
- Agrafiotis, D.K. Stochastic proximity embedding. J. Comput. Chem.
**2003**, 24, 1215–1221. [Google Scholar] [CrossRef] - Cardoso, J.F. High-Order Contrasts for Independent Component Analysis. Neural Comput.
**1999**, 11, 157–192. [Google Scholar] [CrossRef] - De Haan, G.; Jeanne, V. Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng.
**2013**, 60, 2878–2886. [Google Scholar] [CrossRef] - Bobbia, S.; Macwan, R.; Benezeth, Y.; Mansouri, A.; Dubois, J. Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognit. Lett.
**2019**, 124, 82–90. [Google Scholar] [CrossRef] - Mittal, A.; Moorthy, A.K.; Bovik, A.C. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process.
**2012**, 21, 4695–4708. [Google Scholar] [CrossRef] [PubMed] - Venkatanath, N.; Praneeth, D.; Bh, M.C.; Channappayya, S.S.; Medasani, S.S. Blind image quality evaluation using perception based features. In Proceedings of the 2015 Twenty First National Conference on Communications (NCC), Mumbai, India, 27 February–1 March 2015; pp. 1–6. [Google Scholar]
- Fouad, R.M.; Omer, O.A.; Aly, M.H. Optimizing Remote Photoplethysmography Using Adaptive Skin Segmentation for Real-Time Heart Rate Monitoring. IEEE Access
**2019**, 7, 76513–76528. [Google Scholar] [CrossRef] - Lewandowska, M.; Rumiński, J.; Kocejko, T.; Nowak, J. Measuring pulse rate with a webcam—A non-contact method for evaluating cardiac activity. In Proceedings of the 2011 Federated Conference on Computer Science and Information Systems (FedCSIS), Szczecin, Poland, 18–21 September 2011; pp. 405–410. [Google Scholar]
- Mannapperuma, K.; Holton, B.D.; Lesniewski, P.J.; Thomas, J.C. Performance limits of ICA-based heart rate identification techniques in imaging photoplethysmography. Physiol. Meas.
**2014**, 36, 67. [Google Scholar] [CrossRef] [Green Version] - McDuff, D.; Gontarek, S.; Picard, R. Remote measurement of cognitive stress via heart rate variability. In Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014, Chicago, IL, USA, 26–30 August 2014; pp. 2957–2960. [Google Scholar] [CrossRef]
- Chen, J.; Patel, V.M.; Liu, L.; Kellokumpu, V.; Zhao, G.; Pietikäinen, M.; Chellappa, R. Robust local features for remote face recognition. Image Vis. Comput.
**2017**, 64, 34–46. [Google Scholar] [CrossRef] [Green Version] - Verkruysse, W.; Svaasand, L.O.; Nelson, J.S. Remote plethysmographic imaging using ambient light. Opt. Express
**2008**, 16, 21434. [Google Scholar] [CrossRef] [Green Version] - Viola, P.; Jones, M. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA, 8–14 December 2021; Volume 1, p. I. [Google Scholar]
- Baker, S.; Matthews, I. Lucas-Kanade 20 Years On: A Unifying Framework. Int. J. Comput. Vis.
**2004**, 56, 221–255. [Google Scholar] [CrossRef] - Bin Abdul Rahman, N.A.; Wei, K.C.; See, J. Rgb-h-cbcr Skin Colour Model for Human Face Detection; Faculty of Information Technology, Multimedia University: Selangor, Malaysia, 2007; Volume 4. [Google Scholar]
- Holton, B.D.; Mannapperuma, K.; Lesniewski, P.J.; Thomas, J.C. Signal recovery in imaging photoplethysmography. Physiol. Meas.
**2013**, 34, 1499. [Google Scholar] [CrossRef] [PubMed] - Tarvainen, M.P.; Ranta-aho, P.O.; Karjalainen, P.A. An advanced detrending method with application to HRV analysis. IEEE Trans. Biomed. Eng.
**2002**, 49, 172–175. [Google Scholar] [CrossRef] - Tarassenko, L.; Villarroel, M.; Guazzi, A.; Jorge, J.; Clifton, D.A.; Pugh, C. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol. Meas.
**2014**, 35, 807. [Google Scholar] [CrossRef] - Jain, M.; Deb, S.; Subramanyam, A.V. Face video based touchless blood pressure and heart rate estimation. In Proceedings of the 2016 IEEE 18th International Workshop on Multimedia Signal Processing, MMSP 2016, Montreal, QC, Canada, 21–23 September 2016. [Google Scholar] [CrossRef]
- Poh, M.Z.; McDuff, D.J.; Picard, R.W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express
**2010**, 18, 10762–10774. [Google Scholar] [CrossRef] - Bousefsaf, F.; Maaoui, C.; Pruski, A. Peripheral vasomotor activity assessment using a continuous wavelet analysis on webcam photoplethysmographic signals. Bio-Med. Mater. Eng.
**2016**, 27, 527–538. [Google Scholar] [CrossRef] - Wu, B.F.; Chu, Y.W.; Huang, P.W.; Chung, M.L.; Lin, T.M. A motion robust remote-PPG approach to driver’s health state monitoring. Lect. Notes Comput. Sci. (Incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinform.)
**2017**, 10116 LNCS, 463–476. [Google Scholar] [CrossRef] - Schäfer, A.; Vagedes, J. How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram. Int. J. Cardiol.
**2013**, 166, 15–29. [Google Scholar] [CrossRef] [PubMed] - Elgendi, M.; Norton, I.; Brearley, M.; Abbott, D.; Schuurmans, D. Systolic Peak Detection in Acceleration Photoplethysmograms Measured from Emergency Responders in Tropical Conditions. PLoS ONE
**2013**, 8, e76585. [Google Scholar] [CrossRef] [Green Version] - Tarvainen, M.P.; Niskanen, J.P.; Lipponen, J.A.; Ranta-Aho, P.O.; Karjalainen, P.A. Kubios HRV–heart rate variability analysis software. Comput. Methods Programs Biomed.
**2014**, 113, 210–220. [Google Scholar] [CrossRef] [PubMed] - Tulppo, M.P.; Makikallio, T.H.; Takala, T.; Seppanen, T.; Huikuri, H.V. Quantitative beat-to-beat analysis of heart rate dynamics during exercise. Am. J. Physiol.-Heart Circ. Physiol.
**1996**, 271, H244–H252. [Google Scholar] [CrossRef] - Nardelli, M.; Greco, A.; Bolea, J.; Valenza, G.; Scilingo, E.P.; Bailon, R. Reliability of lagged poincaré plot parameters in ultrashort heart rate variability series: Application on affective sounds. IEEE J. Biomed. Health Inform.
**2017**, 22, 741–749. [Google Scholar] [CrossRef] [PubMed] - Lilliefors, H.W. On the Kolmogorov-Smirnov test for normality with mean and variance unknown. J. Am. Stat. Assoc.
**1967**, 62, 399–402. [Google Scholar] [CrossRef] - Zhang, B.; Sennrich, R. Root mean square layer normalization. Adv. Neural Inf. Process. Syst.
**2019**, 32. [Google Scholar] - Shoushan, M.M.; Reyes, B.A.; Mejia Rodriguez, A.R.; Chong, J.W. Contactless Monitoring of Heart Rate Variability During Respiratory Maneuvers. IEEE Sens. J.
**2022**, 22, 14563–14573. [Google Scholar] [CrossRef] - Breslow, N.E. Lessons in biostatistics. Past Present Future Stat. Sci.
**2014**, 25, 335–347. [Google Scholar] [CrossRef] - Hartmann, V.; Liu, H.; Chen, F.; Qiu, Q.; Hughes, S.; Zheng, D. Quantitative comparison of photoplethysmographic waveform characteristics: Effect of measurement site. Front. Physiol.
**2019**, 10, 198. [Google Scholar] [CrossRef] - Nardelli, M.; Vanello, N.; Galperti, G.; Greco, A.; Scilingo, E.P. Assessing the quality of heart rate variability estimated from wrist and finger ppg: A novel approach based on cross-mapping method. Sensors
**2020**, 20, 3156. [Google Scholar] [CrossRef] - Park, S.K.; Kang, S.J.; Im, H.S.; Cheon, M.Y.; Bang, J.Y.; Shin, W.J.; Choi, B.M.; Youn, M.O.; Kim, Y.K.; Hwang, G.S.; et al. Validity of Heart Rate Variability Using Poincare Plot for Assessing Vagal Tone during General Anesthesia. Korean J. Anesthesiol.
**2005**, 49, 765–770. [Google Scholar] [CrossRef] - Keute, M.; Machetanz, K.; Berelidze, L.; Guggenberger, R.; Gharabaghi, A. Neuro-cardiac coupling predicts transcutaneous auricular vagus nerve stimulation effects. Brain Stimul.
**2021**, 14, 209–216. [Google Scholar] [CrossRef] - Shaffer, F.; Meehan, Z.M.; Zerr, C.L. A critical review of ultra-short-term heart rate variability norms research. Front. Neurosci.
**2020**, 14, 594880. [Google Scholar] [CrossRef] - He, X.; Zhao, M.; Bi, X.; Sun, L.; Yu, X.; Zhao, M.; Zang, W. Novel strategies and underlying protective mechanisms of modulation of vagal activity in cardiovascular diseases. Br. J. Pharmacol.
**2015**, 172, 5489–5500. [Google Scholar] [CrossRef] [Green Version] - Dangardt, F.; Volkmann, R.; Chen, Y.; Osika, W.; Mårild, S.; Friberg, P. Reduced cardiac vagal activity in obese children and adolescents. Clin. Physiol. Funct. Imaging
**2011**, 31, 108–113. [Google Scholar] [CrossRef] - Clamor, A.; Lincoln, T.M.; Thayer, J.F.; Koenig, J. Resting vagal activity in schizophrenia: Meta-analysis of heart rate variability as a potential endophenotype. Br. J. Psychiatry
**2016**, 208, 9–16. [Google Scholar] [CrossRef] [Green Version] - Phung, S.L.; Chai, D.; Bouzerdoum, A. Adaptive skin segmentation in color images. In Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, Proceedings (ICASSP’03), Hong Kong, China, 6–10 April 2003; Volume 3, p. III-353. [Google Scholar]
- Niu, Y.; Zhong, Y.; Guo, W.; Shi, Y.; Chen, P. 2D and 3D image quality assessment: A survey of metrics and challenges. IEEE Access
**2018**, 7, 782–801. [Google Scholar] [CrossRef]

**Figure 1.**Experimental setup for the data acquisition, including the webcam-based videos to extract the iPPG signal and the ground truth PPG signal acquired using a finger clip pulse oximeter. A frame taken from one of the UBFC-RPPG videos is shown in figure [24].

**Figure 2.**Flow chart of the four steps of blood volume pulse signal extraction using imaging photoplethysmography. A frame taken from one of the UBFC-RPPG videos is shown in the figure [24], the result of face detection is shown in green.

**Figure 3.**An example of an iPPG signal extracted through the POS method from the video clip of the first subject of the UBFC-RPPG datset (111.6 bpm were found with both iPPG and finger clip PPG monitoring techniques). A frame taken from the video of the first subject of the UBFC-RPPG dataset is shown in the figure [24].

**Figure 4.**Bland−Altman plot of the averaged PR values obtained from iPPG extracted through the POS method and the finger clip PPG.

iPPG Extraction Methods | Short Description |
---|---|

Green–Red Difference (GRD) | iPPG is estimated by the green signal, while the red signal is considered as containing artifacts [17] |

Adaptive Green–Red Difference (AGRD) | An adaptive color difference operation between the green and red channels is applied to reduce motion artifacts [18] |

Principal Component Analysis (PCA) | The most relevant information of RGB data is expressed as a set of new orthogonal variables, called principal components [19] |

Independent Component Analysis (ICA) | RGB signals are decomposed by means of blind source separation and the component with the most prominent peak in the PR bandwith is chosen according to [22] |

Chrominace-Based (CHROM) | A model of PPG-induced variations in color intensities is employed to improve motion robustness [23] |

Plane-Orthogonal-to-Skin (POS) | Improved version of CHROM method which uses a projection plane orthogonal to the skin tone for pulse extraction [13] |

Laplacian Eigenmap (LE) | Unfolds data distribution in a hyperdimensional space in order to reduce dimensionality [20] |

Stochastic Proximity Embedding (SPE) | Generates an one-dimensional Euclidean embedding out of the RGB-data, where the similarities between the related observations are preserved [21] |

PRV Metrics | Unit | Description |
---|---|---|

Time domain | ||

PR | 1/min | Average pulse rate |

RMSSD | ms | Root mean square of successive IBI interval differences |

SDNN | ms | Standard deviation of NN intervals |

TI | - | Integral of NN interval histogram divided by its height |

TINN | ms | Baseline width of the NN interval histogram |

Frequency domain | ||

VLF | ms^{2} | Absolute power of the very-low-frequency band (0.0033–0.04 Hz) |

LF | ms^{2} | Absolute power of the low-frequency power (0.04–0.15 Hz) |

HF | ms^{2} | High-frequency power (0.15–0.4 Hz) |

LF/HF | - | Ratio of LF-to-HF absolute power |

LFnu | n.u. | Relative power in the low-frequency band |

HFnu | n.u. | Relative power in the high-frequency band in normal units |

Nonlinear domain | ||

SD1 | ms | Poincaré plot standard deviation perpendicular to the line of identity |

SD2 | ms | Poincaré plot standard deviation along the line of identity |

SD1/SD2 | - | Ratio of SD1-to-SD2 standard deviations |

**Table 3.**Spearman’s correlation coefficients and corresponding p-values calculated between the values of PRV features extracted through the eight iPPG methods and with the wearable sensor. Bold indicates the lowest NRMSE for each PRV feature.

Spearman Correlation Coefficient ($\mathit{\rho}$) and Statistical Significance (p-Value) | |||||||||
---|---|---|---|---|---|---|---|---|---|

GRD | AGRD | PCA | LE | SPE | ICA | CHROM | POS | ||

Time domain | |||||||||

PR | $\rho $ p-val | 0.980 $1.48\times {10}^{-29}$ | 0.957 $3.76\times {10}^{-23}$ | 0.583 $5.03\times {10}^{-5}$ | 0.437 0.004 | 0.336 0.03 | 0.916 $2.05\times {10}^{-17}$ | 0.971 $1.61\times {10}^{-26}$ | 0.994$2.61\times {10}^{-40}$ |

RMSSD | $\rho $ p-val | 0.376$1.40\times {10}^{-2}$ | 0.330 0.03 | −0.048 0.76 | −0.009 0.95 | 0.168 0.23 | 0.253 0.11 | 0.3760.01 | 0.374 0.01 |

SDNN | $\rho $ p-val | 0.601 $2.56\times {10}^{-5}$ | 0.585 $4.77\times {10}^{-5}$ | 0.278 0.07 | −0.017 0.92 | 0.336 0.03 | 0.676 $9.20\times {10}^{-7}$ | 0.768 $3.01\times {10}^{-9}$ | 0.818$3.63\times {10}^{-11}$ |

TI | $\rho $ p-val | 0.477 $1.41\times {10}^{-3}$ | 0.471 $1.65\times {10}^{-3}$ | 0.202 0.20 | 0.147 0.15 | 0.142 0.37 | 0.530 $3.09\times {10}^{-4}$ | 0.623$1.05\times {10}^{-5}$ | 0.592 $3.64\times {10}^{-5}$ |

TINN | $\rho $ p-val | 0.340 $2.74\times {10}^{-2}$ | 0.331 $3.21\times {10}^{-2}$ | −0.085 0.59 | −0.153 0.33 | 0.167 0.37 | 0.416 $6.15\times {10}^{-3}$ | 0.482 $1.23\times {10}^{-3}$ | 0.510$5.63\times {10}^{-4}$ |

Frequency domain | |||||||||

VLF-pow | $\rho $ p-val | 0.719 $8.10\times {10}^{-8}$ | 0.687 $5.13\times {10}^{-7}$ | 0.021 0.90 | −0.113 0.47 | 0.192 0.22 | 0.915 $2.55\times {10}^{-17}$ | 0.935 $1.13\times {10}^{-19}$ | 0.939$3.75\times {10}^{-20}$ |

LF-pow | $\rho $ p-val | 0.391 $3.91\times {10}^{-1}$ | 0.356 0.02 | 0.267 0.09 | −0.043 0.79 | 0.126 0.43 | 0.659 $2.12\times {10}^{-6}$ | 0.831 $9.22\times {10}^{-12}$ | 0.903$2.69\times {10}^{-16}$ |

HF-pow | $\rho $ p-val | 0.064 0.69 | −0.040 0.80 | −0.102 0.52 | −0.096 0.55 | −0.103 0.52 | −0.09 0.57 | 0.3480.02 | 0.134 0.40 |

LF/HF | $\rho $ p-val | −0.222 0.16 | −0.0925 0.56 | 0.412 $6.78\times {10}^{-3}$ | 0.025 0.88 | 0.178 0.26 | −0.098 0.54 | −0.3130.04 | −0.192 0.23 |

HFnu | $\rho $ p-val | 0.340 0.03 | 0.142 0.37 | 0.035 0.82 | −0.179 0.26 | 0.028 0.86 | 0.286 0.07 | 0.478 $1.38\times {10}^{-3}$ | 0.493$9.09\times {10}^{-4}$ |

LFnu | $\rho $ p-val | 0.325 0.04 | 0.134 0.40 | 0.040 0.80 | −0.170 0.28 | 0.029 0.86 | 0.278 0.07 | 0.477 $1.42\times {10}^{-3}$ | 0.499$7.75\times {10}^{-4}$ |

Nonlinear domain | |||||||||

SD1 | $\rho $ p-val | 0.384$1.19\times {10}^{-2}$ | 0.330 0.03 | −0.048 0.76 | −0.013 0.93 | 0.160 0.31 | 0.252 0.11 | 0.376 0.01 | 0.379 0.01 |

SD2 | $\rho $ p-val | 0.687 $5.00\times {10}^{-7}$ | 0.664 $1.67\times {10}^{-6}$ | 0.327 0.03 | −0.072 0.65 | 0.365 0.02 | 0.816 $4.55\times {10}^{-11}$ | 0.882 $1.17\times {10}^{-14}$ | 0.936$8.85\times {10}^{-20}$ |

SD1/SD2 | $\rho $ p-val | 0.632 $7.12\times {10}^{-6}$ | 0.588 $4.30\times {10}^{-5}$ | 0.032 0.84 | −0.136 0.39 | 0.100 0.53 | 0.689 $4.62\times {10}^{-7}$ | 0.734$3.28\times {10}^{-8}$ | 0.698 $2.74\times {10}^{-7}$ |

**Table 4.**Normalizedroot mean square errors (NRMSE) calculated between the values of PRV features extracted through the eight iPPG methods and with the wearable sensor. Bold indicates the best performance in the correlation tests for each PRV feature.

Normalized Root Mean Square Error (NRMSE) | ||||||||
---|---|---|---|---|---|---|---|---|

GRD | AGRD | PCA | LE | SPE | ICA | CHROM | POS | |

Time domain | ||||||||

PR | 0.0393 | 0.0466 | 0.158 | 0.182 | 0.243 | 0.0348 | 0.0278 | 0.014 |

RMSSD | 0.179 | 0.197 | 0.380 | 0.392 | 0.379 | 0.225 | 0.221 | 0.228 |

SDNN | 0.176 | 0.189 | 0.587 | 0.726 | 0.605 | 0.148 | 0.144 | 0.099 |

TI | 0.210 | 0.226 | 0.671 | 0.676 | 0.720 | 0.194 | 0.178 | 0.163 |

TINN | 0.261 | 0.237 | 0.553 | 0.724 | 0.750 | 0.200 | 0.222 | 0.172 |

Frequency domain | ||||||||

VLF-power | 0.152 | 0.112 | 0.518 | 0.473 | 0.216 | 0.0485 | 0.064 | 0.038 |

LF-power | 0.243 | 0.273 | 1.42 | 1.46 | 1.34 | 0.168 | 0.166 | 0.061 |

HF-power | 0.988 | 1.03 | 1.06 | 1.09 | 1.05 | 1.03 | 0.927 | 0.928 |

LF/HF | 0.232 | 0.224 | 0.215 | 0.218 | 0.241 | 0.229 | 0.234 | 0.230 |

HFnu | 0.231 | 0.261 | 0.336 | 0.330 | 0.286 | 0.244 | 0.203 | 0.194 |

LFnu | 0.231 | 0.260 | 0.331 | 0.328 | 0.283 | 0.243 | 0.202 | 0.193 |

Nonlinear domain | ||||||||

SD1 | 0.179 | 0.196 | 0.378 | 0.389 | 0.376 | 0.223 | 0.220 | 0.227 |

SD2 | 0.194 | 0.215 | 0.644 | 0.821 | 0.675 | 0.135 | 0.129 | 0.063 |

SD1/SD2 | 0.202 | 0.207 | 0.245 | 0.298 | 0.270 | 0.192 | 0.205 | 0.191 |

**Table 5.**Bland–Altman Analysis results in terms of mean differences $\overline{d}$ and 95% confidence intervals $[\overline{d}\pm 1.96SD]$ for each iPPG method when compared with the ground truth.

Bland–Altman Analysis | ||||||||
---|---|---|---|---|---|---|---|---|

GRD | AGRD | PCA | LE | SPE | ICA | CHROM | POS | |

Time domain | ||||||||

PR | −0.474 | −1.14 | −4.53 | −0.041 | 13.2 | −1.90 | 1.05 | −0.186 |

[1/min] | [−9.81,8.86] | [−12.7,10.4] | [−34.7,25.6] | [−34.2,34.1] | [−28.9,55.3] | [−14.0,10.2] | [−9.62,11.7] | [−4.05,3.68] |

RMSSD | −3.37 | −5.97 | 29.4 | 31.9 | 29.7 | −8.89 | −13.5 | −17.3 |

[ms] | [−44.3,37.6] | [−49.3,37.4] | [−42.7,102] | [−29,92.8] | [−18.3,77.8] | [−56.6,38.8] | [−54.8,27.9] | [−58.4,23.8] |

SDNN | 8.65 | 8.99 | 47.6 | 59.6 | 47.8 | 1.55 | 1.11 | −6.18 |

[ms] | [−30.3,47.6] | [−30.1,48.1] | [−33,128] | [−35.8,155] | [−21.6,117] | [−32.3,35.4] | [−42.2,44.4] | [−25.5,13.1] |

TI | 0.374 | 0.465 | 2.57 | 2.91 | 2.97 | 0.210 | 0.149 | −0.208 |

[−2.10,2.85] | [−2.07,3.00] | [−2.21,7.34] | [−1.28,7.10] | [−0.16,6.09] | [−2.25,2.67] | [−1.84,2.14] | [−1.72,1.30] | |

TINN | 0.035 | 0.030 | 0.147 | 0.198 | 0.180 | 0.004 | 0.002 | −0.024 |

[ms] | [−0.14,0.21] | [−0.12,0.18] | [−0.11,0.40] | [−0.08,0.48] | [−0.07,0.43] | [−0.14,0.15] | [−0.18,0.18] | [−0.13,0.08] |

Frequency domain | ||||||||

VLFpow | 161$\times 10$ | 511 | 207$\times 10$ | 443$\times 10$ | 147$\times 10$ | -200 | 554 | −270 |

[ms${}^{2}$] | [−176,208]$\times {10}^{2}$ | [−296,398]$\times 10$ | [−60.1,101]$\times {10}^{2}$ | [128,216]$\times {10}^{2}$ | [−80.0,109]$\times {10}^{2}$ | [−221,181]$\times 10$ | [−566,677]$\times 10$ | [−304,250]$\times 10$ |

LFpow | 597 | 792 | 510$\times 10$ | 514$\times 10$ | 454$\times 10$ | 308 | 452 | 128 |

[ms${}^{2}$] | [−204,323]$\times 10$ | [−228,387]$\times 10$ | [−120,212]$\times {10}^{2}$ | [−88.4,191]$\times {10}^{2}$ | [−214,305]$\times {10}^{2}$ | [−175,236]$\times 10$ | [−197,288]$\times 10$ | [−525,782] |

HFpow | −243$\times {10}^{3}$ | −254$\times {10}^{3}$ | −261$\times {10}^{3}$ | −268$\times {10}^{3}$ | −259$\times {10}^{3}$ | −254$\times {10}^{3}$ | −228$\times {10}^{3}$ | −228$\times {10}^{3}$ |

[ms${}^{2}$] | [−433,−53.7]$\times {10}^{3}$ | [−449,−60]$\times {10}^{3}$ | [−456,21.2]$\times {10}^{3}$ | [−462,−74.1]$\times {10}^{3}$ | [−457,−60.3]$\times {10}^{3}$ | [−444,−63.2]$\times {10}^{3}$ | [−408,−47.7]$\times {10}^{3}$ | [−409,−48]$\times {10}^{3}$ |

LF/HF | −1.47 | −1.44 | −1.57 | −1.40 | −1.69 | −1.47 | −1.41 | −1.45 |

[−4.40,1.47] | [−4.48,1.60] | [−5.09,1.96] | [−4.27,1.46] | [−5.32,1.94] | [−4.35,1.40] | [−4.07,1.25] | [−4.30,1.40] | |

HFnu | 4.65 | 1.08 | 1.20 | −3.18 | 3.58 | 3.87 | −0.472 | −1.26 |

[n.u.] | [−36.0,45.3] | [−47.0,49.1] | [−55.8,58.2] | [−62.7,56.4] | [−47.7,54.8] | [−37.9,45.7] | [−36.0,35.1] | [−35.7,33.1] |

LFnu | −4.60 | −0.953 | −0.984 | 3.39 | −3.41 | −3.73 | 0.545 | 1.31 |

[n.u.] | [−45.7,36.5] | [−49.2,47.3] | [−58.0,56.1] | [−56.5,63.2] | [−54.8,48.0] | [−45.7,38.2] | [−35.2,36.3] | [−33.2,35.9] |

Nonlinear domain | ||||||||

SD1 | −2.37 | −4.25 | 21.1 | 22.8 | 21.1 | −6.35 | −9.58 | −12.3 |

[ms] | [−31.6,26.8] | [−35.2,26.7] | [−30.7,72.9] | [−21.0,66.6] | [−13.4,55.7] | [−40.4,27.7] | [−39.1,19.9] | [−41.7,17.0] |

SD2 | 15.9 | 17.9 | 67.1 | 85.0 | 67.7 | 7.33 | 7.42 | −2.47 |

[ms] | [−36.7,68.5] | [−34.6,70.5] | [−47.5,182] | [−56.2,226] | [−35.2,171] | [−32.3,46.9] | [−51.9,66.7] | [−19.1,14.2] |

SD1/SD2 | −0.163 | −0.207 | −0.199 | −0.243 | -0.238 | −0.169 | −0.177 | −0.169 |

[−0.58,0.26] | [−0.59,0.18] | [−0.78,0.38] | [−0.85,0.37] | [−0.80,0.33] | [−0.52,0.18] | [−0.49,0.14] | [−0.49,0.16] |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

van Es, V.A.A.; Lopata, R.G.P.; Scilingo, E.P.; Nardelli, M.
Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring. *Sensors* **2023**, *23*, 1505.
https://doi.org/10.3390/s23031505

**AMA Style**

van Es VAA, Lopata RGP, Scilingo EP, Nardelli M.
Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring. *Sensors*. 2023; 23(3):1505.
https://doi.org/10.3390/s23031505

**Chicago/Turabian Style**

van Es, Valerie A. A., Richard G. P. Lopata, Enzo Pasquale Scilingo, and Mimma Nardelli.
2023. "Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring" *Sensors* 23, no. 3: 1505.
https://doi.org/10.3390/s23031505