Robust RPPG Method Based on Reference Signal Envelope to Improve Wave Morphology

Abstract

Remote physiological monitoring has become increasingly important in improving quality of life, with remote photoplethysmography (RPPG) being a popular choice. This paper introduces an envelope–based method for RPPG channels to improve wave morphology of the collected signal based on the reference signal from finger PPG. Using a model consistent with physiological and optical principles, the authors divided the signal into linear superpositions, comprising pulse, constant, and disturbance components. The correlation coefficients were used to calculate a linear combination of Red–Green–Blue (RGB) channels to approximate the envelope shape of the reference PPG signal. Experiments with different light intensities and stability were designed to compare the envelope approximation ability and robustness of the proposed method with some common methods. Analysis of variance demonstrated the stable performance of the envelopment–based approach in most cases. Additionally, it improved the morphology of the Green (G) channel, including changing trends and directions, adjusting wave sizes, reducing noise, and reinforcing details of the single waveform. The envelope–based linear model approach has the ability to flexibly improve RPPG signals, which helps RPPG play a full role in many fields such as medicine.

1. Introduction

Photoplethysmography (PPG) is a progressive optical technique for non–invasively monitoring physiological signals. It utilises the reflection or transmission properties of light to capture the interaction between optics and subcutaneous microscopic vascular tissue, which reflects the pulsation propagated by the heartbeat. PPG–based sensors have enjoyed popularity in recent years due to their advantages of low cost, non–invasive nature, and convenience. They are widely used to measure physiological signals, such as heart–rate variability [1], respiration [2], blood pressure [3], and mental stress [4], and have various applications in scenarios such as driving [5] and clinical practice [6]. Many commercial products [7,8], such as various smartwatches and wristbands, are already available on the market.

Contact PPG generally takes the light transmission mode, such as the finger clip equipment adopted in this paper. Another type of non–contact PPG equiped with a distance camera, called remote PPG (RPPG), takes a reflection model. However, both of these forms are very susceptible to environmental interference and motion artifacts. Various methods have been proposed to address this issue, including Green (G) [9], Green minus Red (G−R) [10], chrominance–based method (CHROM) [11], and plane–orthogonal–to–skin (POS) [12]. They combine Red–Green–Blue (RGB) channels to form the pulse signal in different ways.

The PPG signals contain detailed information in the pulse wave, such as the waveform, amplitude, energy, and envelope. The envelope is more global than any other feature, describing the general tendency of the wave. However, research on the envelope has been relatively limited compared to the other features, despite its relevance to certain physiological characteristics. For instance, Linder and Wendelken [13] detected moderate hypovolemia utilising the top and bottom envelope of the PPG. Sandberg et al. [14] investigated changes in the normalised envelope of the test statistic of PPG signal to predict acute symptomatic intradialytic hypotension. Additionally, Kuwałek et al. [15] used the PPG envelope as one of the characteristics to detect respiratory rate. Despite the potential utility of the envelope feature, more research is needed to fully explore its clinical significance.

As a completely contactless and highly flexible technology, RPPG has been widely applied in physiological monitoring [16], biometrics [17], and other fields in recent years. Especially in the medical domain, RPPG has a strong potential in situations where direct contact with the skin may cause discomfort to patients, or even where contact must be avoided. For example, Kamshilin et al. [18] used the RPPG system to monitor tissue perfusion in open surgery and demonstrated its feasibility, and Schraven et al. [19] paid special attention to perfusion monitoring during flap reconstruction. Differently from traditional laboratory settings, Allado et al. [16,20] evaluated the application of the RPPG system in clinical practice by measuring conventional physiological indicators, further verifying the practicability of RPPG technology. With the development of deep learning techniques, the measurement of routine physiological variables, such as blood pressure [21] and heart rate [22], has become more efficient and accurate. At the same time, more complex and subjective health indicators, such as stress [23] and depression [24], are gaining more attention in modern times.

Li et al. [22] divided the RPPG–based HR measurement methods into two types: signal analysis–based and data–driven. The former is similar to the model–based approach to obtain a more complete RPPG signal; the latter is often targeted at specific physiological variables, such as more accurate breathing rate and heart rate. Haugg et al. [25] also pointed out that RPPG signals are usually compared only to target health information, such as heart rate, not to ground truth PPG signals. They argue that more complex health–related indicators require higher–quality signals, and evaluated the similarity of RPPG signals from several non–deep learning methods to the reference finger PPG signal. Kim et al. [26] realised the degradation of individual waveforms of RPPG signal, using support vector regression (SVR) and deep learning models to recover RPPG signals by corresponding to contact PPG. We directly target the trend shape of finger PPG, using the most basic linear combination of RPPG signal channels to maximise the retention of envelope, a holistic feature. The linear combination method belongs to the category of traditional model–based approaches, which have gained far less popularity in recent years compared to deep learning methods. However, helping to meet the modern requirements for accurate measurement of more complex and diverse health information, it is the simpler and more essential model approach, ensuring higher quality of RPPG signals. Additionally, it is highly compatible with modern deep learning tools.

In the context of modern health monitoring, this paper explores a linear model approach to improve RPPG signals, based on the envelope shape. We proposed an envelope–based method (EB), aiming to identify the optimal combination of original RGB channels in RPPG, with reference to the finger PPG. We adopted the skin light reflection model, similar to the one presented in prior studies [12], to decompose the RPPG signal into a linear combination of pulsation, constant, and disturbance components. Based on the envelope shape of the reference PPG curve, we used correlation coefficients to determine the optimal channel combination for approximation. Furthermore, we conducted experiments under four illumination conditions with different intensities and stability, to evaluate the envelope approximation capabilities of various methods. The results demonstrated that our method retained the envelope shape that was more closely aligned with the benchmark PPG curve compared to other approaches.

2. RPPG Framework

In this section, we present the RPPG processing scheme commonly used today. The scheme involves four main steps: appropriate area tracking, regional pixel processing, colour channel selection, and noise filtering and information extraction. The general flow can be found in Figure 1.

The first step, region of interest (ROI) selection and tracking, is fundamental to obtain high–quality physiological information. Most cameras capture images of faces, and the ROI is usually selected by the VJ method (an efficient real–time face detection algorithm based on new image representation of an original image) [27] or the CSK method (a high–speed tracker that uses the kernel function and fast Fourier transform) [28]. Sungjun Kwon et al. [29] found that the cheeks and forehead were the best choices for ROI on the face. Mohamed et al. [30] achieved better performance by selecting the forehead as the ROI and using the green spectrum of the additive colour space. Adaptive approaches, such as filtering out non–skin pixels [31], have also been proposed. These ROIs are typically selected because of the richness of capillaries. Volkov et al. [32] analysed high–speed video recordings of capillaries in the nail bed and found that the capillary bed can be used as a distributed sensor to monitor the status of deep vessels. Therefore, in our experiments, we selected the finger as the ROI because it also has a high proportion of capillaries.

The integration of spatial pixels is an effective way to reduce illumination variation and system noise. Pixel averaging is the most commonly used method, as proposed by Verkruysse et al. [9] in 2008, which is only suitable for relatively uniform illumination. When light spreads asymmetrically on the skin, it is more efficient to separate the pulse signal based on the spatial redundancy of an image sensor or to rotate the subspace using skin pixel distribution [33]. In this study, we conducted experiments under simple and uniform illumination and adopted the pixel averaging method, which is in line with the basic conditions of use.

For appropriate colour channel, it is crucial to consider the PPG reflection model and skin optical properties. Two types of methods are popular choices for channel selection: statistical techniques such as blind source separation (BSS) and model–based analysis. BSS techniques can extract pulse signals from multiple channels without prior knowledge and filter out some noise pollution such as motion artifacts. Independent Component Analysis (ICA) [34] and Principal Component Analysis (PCA) [35] are typical BSS methods, and their generalisations, such as robust ICA [36] and single–channel ICA [37], have also been proposed. However, the results of these separation methods are only approximations and lack a physiologically convincing explanation.

The model–based approach is a more physiological and principled way of analysing the signal source, taking into account the influence of motion and the environment and retaining the original physiological information. It usually describes the received signal as the sum of the pulsating component, stationary component, and disturbed component. In 2013, de Haan and Jeanne [11] proposed the chrominance model (CHROM) for distinguishing pulsation signals. In the following year, de Haan and Van Leest (Blood Volume Pulse method, PBV) [38] analysed the characteristics of blood volume changes at different wavelengths from both physiological and optical perspectives. In 2017, Wang et al. (POS) [12] proposed a mathematical model for skin light reflection and interpreted BSS, CHROM, and PBV methods with this model. They used a projection plane orthogonal to skin colour to extract the pulse signal.

In this paper, we utilise a similar model that decomposes the signal into a linear combination of variable pulse, fixed constant, and disturbance [12]. This method is supported by sound principles and can retain physiological information to a great extent. Unlike the analysis of chroma, blood volume, and projection space, our approach optimises the linear combination of RGB channels for envelope similarity based on a reference curve. We take the finger PPG as the ground truth in our experiments, but are not limited to it. Our simple but effective approach ensures the integrity of physiological signals while reducing noise disturbance.

The final step of the RPPG processing scheme is noise filtering and signal extraction, the quality of which is dependent on the effectiveness of channel selection in the previous step. If the channel contains a high proportion of valuable physiological information, simple smoothing techniques such as regular smoothing perform well. However, if the channel contains a lower proportion of valuable information, even advanced filtering methods such as band–pass filter [9], moving average filter [34], and wavelet transform denoising [39] may not produce the desired results. Once noise has been filtered out, researchers extract information such as heart rate, respiratory rate, and blood pressure. The capture of information can be accomplished through interpolation, machine learning approaches such as SVR [40] and Neural Network [41], or probabilistic algorithms [42] such as the adaptive hidden Markov model. Time domain and frequency domain analysis methods are also frequently utilised.

In our view, the key to obtaining accurate physiological information lies in the quality of the pulse signal itself. By ensuring that the pulse signal is of high quality, relevant feature extraction becomes a non–issue. This is why we made it our primary goal to obtain a pure pulse signal that also retained the envelope shape. In fact, the envelope shape has been a previously neglected point of pulse signal analysis, but we recognised its importance in improving the quality of RPPG signal. Based on this RPPG framework, we studied the most critical step: how to obtain a high quality pulse signal. To achieve this, we used a simple and effective combination method that retained valuable physiological information while reducing noise disturbance. The following sections provide more detailed information on our approach.

3. Materials and Methods

3.1. Envelope–Based Algorithm

This part details the proposed envelope–based method and its implementation. The RGB channel combination that was closest to the reference curve was obtained by calculating the correlation coefficient. The combination was kept linear to preserve the physiological information of the pulse. In this context, the envelope was defined as the curve that connected the consecutive peaks of a modulated wave, as per the definition of electronic informatics. The MATLAB function ‘envelope’ (parameter setting: sampling number = 30, ‘peak’) was used to obtain the signal envelope in the experiment.

We supposed the finger PPG as the ground truth, which provided a pulse signal with an accurate envelope curve $y\left(t\right)$. Similar to the skin light reflection model in paper [11,12], we could divide each channel into variable, fixed, and interference terms. The observed RGB signals at one pixel p all had similar expressions [12]:

$$x_C\left(t,p\right)=a_{s,C}\left(t,p\right)·s\left(t,p\right)+a_{d,C}\left(t,p\right)·d\left(t,p\right)+n_C\left(t,p\right),$$

(1)

where $C\in \{R,G,B\}$ denotes the colour channels; $x_C(t,p)$ is the channel value at the ROI pixel p; $s\left(t,p\right)$ denotes the pulse signal; $d\left(t,p\right)$ is the non–pulsatile signal, such as ambient light intensity, motion artifact, or the absorption and reflection of bones and muscles; $a_{s,C}\left(t,p\right)$ and $a_{d,C}\left(t,p\right)$ describe the intensity of the signals; and $n_C\left(t,p\right)$ is a zero–mean small noise at one pixel, which is minimal after averaging over the entirety of the ROI pixels. Then, we operated Equation (1) over the ROI pixels and wrote the averaged signal of $x_C(t,p)$ as $x_C\left(t\right)$. Under some relatively stable circumstances, where the ambient illumination was uniform and the subject kept quiet, pulsatile and non–pulsatile components were locally homogeneous in the ROI. This means that $s(t,p)$ and $d(t,p)$ were independent of p, and we could rewrite them as $s\left(t\right)$ and $d\left(t\right)$. Analogously, we considered the same assumptions for the intensities $a_{s,C}(t,p)$ and $a_{d,C}(t,p)$. Therefore, Equation (1) at one pixel can be extended over the pixels in ROI, which follows Equation (2):

$$x_C\left(t\right)=a_{s,C}\left(t\right)·s\left(t\right)+a_{d,C}\left(t\right)·d\left(t\right).$$

(2)

Both pulsatile $s\left(t\right)$ and non–pulsatile $d\left(t\right)$ components are identical for all three channels, while the only difference is their strength. With the homogeneity, we could further assume that the intensity coefficients were constant, that is, $a_{s,C}\left(t\right)=a_{s,C},a_{d,C}\left(t\right)=a_{d,C}$. These constants vary from person to person and can be affected by skin colour, age, and gender.

Different channels may have varying envelopes due to their different absorption and reflection capabilities. In some cases, the envelope shape may differ from that of the finger PPG due to channel sensitivity to noise. We considered the linear combination of the RGB channels, which preserved physiological information while reducing noise. This minimised the damage or loss of the important pulsatile information, since some rough filtering methods may destroy the integrity of pulsatile information.

To figure out the best combination, the phase and frequency between standard $y\left(t\right)$ and observed $x_C\left(t\right)$ first need to be synchronised, which are still represented by the original notations. The combined signal is written as follows:

$$x\left(t\right)=1·x_G\left(t\right)+r·x_R\left(t\right)+b·x_B\left(t\right),$$

(3)

where green channel is fixed, and we adjusted the other two. Futhermore, the base curve $y\left(t\right)$ needs to be standardised, written as $\overline{y}\left(t\right)$.

We used a correlation coefficient to describe the shape similarity. The higher the coefficient, the more similar it is. Thus, the major objective was to maximise the correlation coefficient:

$${\rho }_{\overline{x},\overline{y}}=\frac{(\overline{x}-{\mu }_{\overline{x}},\overline{y})}{{\left|\left|\overline{x}-{\mu }_{\overline{x}}\right|\right|}_2},$$

(4)

where $\overline{x}$ is the target signal; $(·,·)$ is the inner product in Euclidean space; ${\left|\left|·\right|\right|}_2$ denotes $2–$norm; and ${\mu }_{\overline{x}}$ is the mean of $\overline{x}$.

The solution to Equation (4) can be reduced to a quadratic equation of one variable with respect to b, and the extreme points of ${\rho }_{\overline{x},\overline{y}}$ can be solved precisely, where $\overline{b}$ is linear with $\overline{r}$. Then, a rough signal is obtained:

$$\overline{x}\left(t\right)=1·x_G\left(t\right)+\overline{r}·x_R\left(t\right)+\overline{b}·x_B\left(t\right).$$

(5)

Sometimes, it seemed to be chaotic, because we could not reduce the stochastic noise directly.

For example, Figure 2 shows the original green channel $x_G\left(t\right)$, combined signal $\overline{x}\left(t\right)$, smoothed combined signal, and the benchmark curve $y\left(t\right)$. The abscissa in the figure is time (seconds); the uppermost ordinate represents the original value of the green channel, the middle two represent the normalised signal value, and the bottom is the sum of all pixels, meaning that the value of finger PPG signal is very large. Adding the pixel values instead of averaging them preserves the original shape of the signal. The horizontal and vertical coordinates of the following figures are based on this description, and the units are omitted. This does not affect the results of the experiment, because the shape of the signal does not change with units. It is necessary to smooth the combined signal in the last step. We used the basic regularisation smoothing method, and the processing result is shown in the Figure 2. During the calculation, the choice of length parameter of $y\left(t\right)$ and $\overline{x}\left(t\right)$ is also very important to the result, where a segment that is too long or too short is not appropriate. We took the length to be 10 s in this study.

While we received and combined the RGB channels, the illumination condition was recorded by a fixed bright card. Though it was not a real value, it differed only in multiples. Moreover, the card was easy to place and relatively steady. We described the light information of the environment in this way.

This envelope–based method used the ground truth signal to obtain the combination of RGB channels, which retained complete pulse information. It mainly improved the colour channel selection part of the RPPG framework, as in Section 2, and also provided a new perspective and mode for the way the channels were combined. The principle of this method is simple but reasonable, applicable to all kinds of reference signals, and can be flexibly integrated into the operating framework.

3.2. Experiments

3.2.1. Subjects

In total, 12 healthy Asian subjects (11 adults and 1 juvenile, 7 males and 5 females, mean ± SD of age: 36 ± 9 years) participated in this study. None of the subjects had cardiovascular, respiratory, or neurological diseases. Ethical approval was obtained by Tsinghua University Medical Ethics Committee for this study.

3.2.2. Data Collection

We used a Logitech C1000e camera with 1920 × 1080 pixels at a fixed sampling rate of 60 fps. Some additional functions had been turned off, including white balance and automatic focusing. The ground truth signal was obtained by a finger PPG at a sampling frequency of 500 Hz. Each subject was required to sit comfortably and breathe gently at an indoor temperature of 20–25 degrees Celsius. The whole setup is presented in the left panel in Figure 3; the subject stayed still and a small bright card was placed against the palm. Under four different uniform light conditions, we recorded videos for 30 s and synchronously collected signals from finger PPG. Figure 3 shows the view of the camera. The PPG device in the experiment used a probe of 660–905 nm red and infrared wavelengths, model OEM–S3951A, from Med–linket Medical Electronics Co., Ltd. (Shenzhen, China).

3.2.3. Design of Experimental Illumination Conditions

Since some of the literature has shown promising results in a controlled laboratory environment, it is crucial to investigate the performance differences of various algorithms under different working conditions. In the case of RPPG, which is a remote camera system designed to detect exposed skin generally in relaxed conditions without pressure, lighting is one of the biggest challenges. Therefore, we designed four illumination conditions with a stable fill lamp and flashing LED light (see Figure 3e–h), to compare the performance of each algorithm on envelope approximation:

• Low level control, weak fill light with no LED lamp (‘weak fill’);
• High level contol, strong fill light with no LED lamp (‘strong fill’);
• Normal circumstance, only LED lamps without supplementary light source (‘LED’);
• Mixed environment, LED lamps with strong fill light (‘LED + strong’).

Fill lights, which are stable and controlled light sources, are commonly used in research because they minimally affect the results. On the other hand, unstable high–frequency LED lamps (50 Hz in China) are often used in uncontrolled indoor environments as the primary light source. These unstable lamps have a strong interference, resulting in lower signal quality of RPPG compared to fill lamp conditions.

The combination of two light sources helped to show the stability of the environment. Specifically, we used ‘weak fill’ and ‘strong fill’ to represent controlled experimental environments, while ‘LED’ reflected a general indoor lighting environment. Additionally, ‘LED + strong’ involved adding fill light on the basis of ‘LED’. We then tested the envelope approximation effect of each method under these different illumination conditions.

As in Figure 3, two ROIs were selected in the videos. The red rectangle represented the signal of the finger skin, while the green rectangle represented the illumination information. The EB method was then applied to obtain the combined signal based on the finger PPG. The calibration capability and adaptability of several methods were explored in these sets of experiments, as discussed in the following sections.

3.2.4. Compared Methods

We compared the EB method with commonly accepted CHROM [11] and POS [12] methods, as well as G [9], G−R [10], and G−B (green channel minus blue channel) methods. The main comparison metric was the correlation coefficient between the upper envelope of the combined and unprocessed signal, obtained by the ‘envelope’ function in MATLAB, and the groundtruth finger PPG signal curve. Moreover, we also observed the morphological relationships between signals, such as the shape of the envelope and the location of the inflection point.

3.2.5. Statistical Analysis

One–way analysis of variance (ANOVA) was used to compare the differences in correlation coefficients among the methods, particularly the EB, CHROM, and POS methods. The p-value was taken as the criterion of statistical significance, and the conventional value 0.05 was chosen as the threshold. In this study, when the p-value was less than 0.05, it was considered that the performance of the approximate envelope shape between two methods was significantly different. Using the ANOVA test, the envelope–fitting ability of different methods could be distinguished.

4. Results

4.1. Channel Light Information under Different Illumination Conditions

We simulated four different illumination conditions by combining a stable fill light and a high–frequency LED lamp in the experiments. To record the light information of the environment and skin, we selected ROIs in the video for the card and finger (Figure 3). The stability of each channel (RGB) value was measured using the standard deviation, which allowed us to assess the influence of the light source and environment.

Figure 4 shows the light information of finger skin and environment of a subject under different lighting conditions. The results were similar for other subjects.

The stability of environmental light was better under controlled conditions where only fill lights were used, as evidenced by the illumination conditions ‘weak fill’ and ‘strong fill’. Especially in the green and red channels, higher fill light intensity led to significantly better light information of finger skin. However, increasing the brightness on the ‘weak fill’ condition (‘strong fill’) did not improve the instability in the red channel, which may be closely related to skin colour. In general indoor environments (‘LED’), high–frequency LED lamps significantly interfered with the light information of skin and environment. This phenomenon was considerably improved after the introduction of fill light (‘LED + strong’). Across the four groups of light conditions, the green channel of finger skin was the most volatile channel, followed by the blue channel, while the red channel was relatively the most stable.

4.2. Statistical Comparison of Methods

Out of the twelve subjects, one was excluded from analysis because the PPG device did not fix well on their finger, resulting in a large difference in the shape of the PPG signal and green channel. The following results and analyses were obtained from the remaining eleven subjects.

Table 1. Results of ANOVA between EB and other methods. When the p-value was less than 0.05, the two methods were considered to be significantly different.

Illumination Condition	EB vs. CHROM	EB vs. POS
Weak fill	0.0309	0.0210
Strong fill	0.0268	0.0029
LED	0.0360	0.0540
LED + strong	0.0413	0.0021

displays the ANOVA results from comparing the EB method with other methods under four illumination conditions. The EB method showed significant advantages over the CHROM and POS methods in most cases, except for the ‘EB vs. POS’ comparison under LED light condition. Moreover, the EB method performed much better than the POS method when additional fill light was introduced or the intensity of fill light was increased (p-value = 0.0029 for ‘strong fill’, p-value = 0.0021 for ‘LED + strong’).

Figure 5 shows the distribution of envelope correlation obtained by several methods under the four different illumination conditions. It can be observed that, when fill light was added or its intensity was increased, the box width corresponding to most methods was reduced. This indicates that the stability of the environmental light has a significant impact on the effectiveness of the algorithms.

4.3. Analysis of Envelope Morphology

Representative wave patterns from four subjects were selected under the four lighting conditions.

Figure 6 and Figure A1, Figure A2 and Figure A3 show a morphological comparison of the upper envelope between the signal of EB, CHROM, and POS methods and the groundtruth PPG signal, as well as the improvement effect of the EB method on the green channel signal. The left image presents the raw signals (blue line) of EB, CHROM, POS methods, and finger PPG and their upper envelopes (orange line), from top to bottom. In the right image, the top curve represents the original green channel signal, the middle one is the signal obtained by the EB method, and the bottom one is the benchmark finger PPG signal.

Similar to the ‘weak fill’ condition, Figure 7 (and Figure A4, Figure A5 and Figure A6), Figure 8 (and Figure A7, Figure A8 and Figure A9), and Figure 9 (and Figure A10, Figure A11 and Figure A12) show the typical waveforms from the four subjects in the ‘strong fill’, ‘LED’, and ‘LED + strong’ conditions, respectively.

5. Discussion

5.1. Superior Performance of EB Method under Different Illumination Conditions

The ANOVA results between the EB and CHROM/POS methods (Table 1) demonstrate that the EB method performs superior envelope approximation ability across all four illumination conditions. Notably, in the ‘strong fill’ (p-value = 0.0029) and ‘LED + strong’ (p-value = 0.0021) conditions, where environmental light is more stable, the performance of the EB method is significantly better than that of the POS method.

Based on the analysis of the envelope correlation distribution of EB, G−R, G−B, G, CHROM, and POS methods, as well as the characteristics of the environmental light, it can be confirmed that the EB method is superior and that the environment has a non–negligible influence. The box corresponding to the EB method is consistently at the top of the figure, indicating its ability to achieve better envelope approximation. Additionally, the width of the box is relatively narrow, illustrating the robustness of the EB method. Furthermore, the more stable the environment light is, the more robust the EB method becomes.

The envelope approximation ability and stability of G−R, G−B, and G methods are found to be challenging compared to the EB method. The CHROM and POS methods are slightly more stable, but the envelope obtained is different from the benchmark PPG signal. The box corresponding to POS method is observed to be the narrowest around the midline, which is in line with the results of the morphological discussion below. POS being a projection algorithm pulls the signal back to the horizontal direction.

5.2. Advantages and Improvements of EB Method on Envelope Morphology

Based on the representative waveforms of four subjects given in the previous section, the envelope morphology characteristics of EB method over CHROM and POS methods and the improvement of EB method over the original green channel signal are discussed here.

The signal obtained by the EB method retains the envelope similarity with the PPG finger curve most of the time under different illumination conditions, except for subject 6 in Figure A4, subject 9 in Figure A9, and subject 12 in Figure A12. These exceptions may be related to the dissimilarity of the envelope between the original green channel and the PPG signal.

The CHROM method exhibits significant instability, with nearly half of the signal curves being inconsistent with the PPG signal. This inconsistency is apparent in the curves of subject 5 in Figure 6, subject 5 in Figure 7, subject 7 in Figure A8, and subject 5 in Figure A10, as they differ significantly from the PPG signal in the envelope trend and fluctuation position. However, in some cases, CHROM can achieve good results, such as the signal of subject 6 in Figure A4. It had smaller gaps and more complete individual waveforms than that obtained from EB. On the other hand, the POS method has a noticeable characteristic of detrending the signal, resulting in a flattened shape of the envelope. This behavior is consistent with the features of POS box–plots.

The EB method improves the original green channel signal in four ways: altering the curve’s trend and direction, adjusting the fluctuation scale, reducing noise, and enhancing the details of a single wave. In cases where there is a significant difference between the green channel and PPG signals, the EB method can correct the signal shape, as seen in subject 6 in Figure A4 and subject 2 in Figure 9. The EB method can also eliminate signal flicker and adjust the magnitude of fluctuations, as seen for subject 5 in Figure 6 and subject 9 in Figure A11. Under ‘weak’ light conditions, the green channel signal is often noisy, and the EB method can effectively reduce noise, as seen in the signals of the four subjects in Figure 6 and Figure A1, Figure A2 and Figure A3. Additionally, the EB method can complement the details of a single waveform, such as for subject 12 in Figure A6 and subject 5 in Figure A10.

5.3. Limitations

The study has certain limitations that should be taken into account. Firstly, the sample size was relatively small, and the age distribution was limited. Moreover, all the participants were of Asian descent and had similar skin tones. None of the subjects had cardiovascular or respiratory diseases. Therefore, it should be considered to expand the sample size and diversify the subject categories in future experiments.

5.4. Future Work

The effectiveness of the EB method in this study provides valuable insights into interpreting PPG signals. Future experiments could further investigate the impact of lighting, skin tone, and environmental factors on PPG (RPPG) signals. Specifically, we are concerned about several issues: the specific relationship between the absorption and reflection of different light by the skin; the effect of skin colour on the former; the correspondence between channel coefficients and environment illumination; application scenarios; and the choice of reference curve, since even the shape of PPG signals on the left and right fingers may differ. These investigations will advance the capabilities of RPPG systems in measuring pulse waves. Furthermore, we aim to explore the physiological sources of PPG (RPPG) signals in future research.

6. Conclusions

In recent years, non–contact physiological monitoring systems have become of wide concern. The representative RPPG system has played an important role in medical, social, security, and other fields. Especially in the context of modern health, it is capable of real–time intraoperative monitoring of perfusion [18,19], identification of stress [23] and depression [24] in contemporary life, and accurate estimation of routine physiological variables such as heart rate when combined with deep learning tools [22]. However, the low quality of signals from RPPG is a problem for it to completely move from the laboratory to the market.

PPG systems are contactless but non–invasive and have high signal quality compared to RPPG. However, little research [26] has calibrated RPPG signals with reference to PPG signals. Most of the RPPG signals are compared to the target physiological variables, not the ground truth PPG [25]. This study proposed a method to improve the RPPG signal based on the envelope shape of the reference PPG signal. The principle of the method is an RGB linear model, which decomposes the observed signal into a linear combination of pulse–related, constant, and noise–disturbed components [12]. Through the correlation coefficient, the optimal RGB combination is obtained so that RPPG signal has the best envelope approximation effect.

Our proposed EB method outperformed the G−R [10], G−B, G [9], CHROM [11], and POS [12] methods under four designed lighting conditions, providing a stable and efficient approximation effect. In addition, it also improved the morphology of RPPG signals on the envelope in four aspects: changing the trend and direction of the curve, adjusting the wave scale, reducing noise, and enhancing the detail of the individual waveform. The impact of environmental stability on each method was also discussed. The results demonstrate the effectiveness of the proposed envelope–based method and its potential to enhance the accuracy of RPPG signals in various applications. In the future, we will expand the sample size, widen the sample types, conduct experiments in more diverse scenarios, and excavate the individualisation law of the combination coefficient.